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Abstract 

The goal of this thesis is to apply the computational approach to motor learning, 
Le., describe the constraints tliat enable performance Improvement with experience 
and also the constraints that must be satisfied by a motor learning system, describe 
what ia being computed in order to achieve learning;, and why it is being computed., 
The particular tasks used to assess motor learning are loaded and unloaded free 
arm movement, and the thesis includes work on rigid body load estimation, arm 
model estimation, optimal filtering for model parameter estimation, and trajectory 
learning from practice. Learning algorithms have been developed and implemented 
in the context- of robot arm control. 

The thesis demonstrates some of the roles of knowledge in Seaming, Powerful 
generalisations' can be made on the basis of knowledge of system structure, as is 
demonstrated in the load and arm model estimation algorithms, Improving the 
performance of parameter estimation algorithms used in learning involves knowledge 
of the measurement noise characteristics, as is shown in the derivation of optimal, 
filters. Using trajectory errors to correct commands requires knowledge of how 
command errors are transformed into performance errors, i.E., an accurate model of 
the dynamics of the controlled system, as is demonstrated in the trajectory learning 
work. The performance demonstrated by the algorithms developed in this thesis 
should be compared with algorithms that use less knowledge, such as table based 
schemes to learn arm dynamics, previous single trajectory learning algorithms, and 
much of traditional adaptive control, 
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Chapter 1 


Introduction 


Learning is currently an Important distinguishing characteristic between biological 
systems and machines. Humans and animals tend to Improve their motor perfor¬ 
mance with experience* while machines often repeat the same errors, day in and day 
out. An important step towards understanding ourselves, and also building more 
useful machines, is to begin to understand how learning is achieved. 

The goal of this thesis is to apply the computational approach (Marr, 1977; Marr, 
1992; Hildreth and Hollerhach, 1&$5) to motor learning. The point of view taken is 
that learning is an information processing problem: a learning algorithm modifies 
commands using information from previous experience so as to improve performance. 
A proposed methodology for computational neuroscience and artificial intelligence 
includes the following steps: (Marr, 1977; Winston, 1984) 

1 . Identify a particular information processing problem or task. 

2. Formulate a computational theory: Expose constraints that enable [or restrict] 
performance, describe what is being computed and why, select or devise ap¬ 
propriate representations, define procedures to solve problem. 

3. Devise and implement actual algorithms to solve problem. 
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■b Verify and evaluate the performance of the implementation via experiments. 


The particular tasks used to assess motor 'earning are loaded and unloaded free 
arm movement. The computational theory developed applies model based feedfor¬ 
ward control to the arm control problem, and involves elements of control theory and 
system identification. In order to effectively use mode] based control one most have 
an accurate model, and the first half of the thesis describes how to build models of 
particular types of loads and arms, One constraint that Is heavily relied upon is that 
the arms and loads described in this thesis can be adequately modelled using rigid 
body dynaintcs. This allows us to use models with small numbers of parameters and 
to make powerful generalizations between dissimilar movementS- 

The implementation of the model building procedures reveals that good models 
can he identified quickly and are useful for control. However, the models us-Ed to 
represent the arm and load dynamics have limited degrees of freedom, and cannot 
represent the frill complexity of the true dynamics. The second half of the thesis 
describes how a command can be relined for a particular movement on the basis of 
practice^ allowing representation of fine details of the dynamics. The computational 
theory for the learning from practice described here involves making Explicit what, 
the learning operator that maps performance errors to command corrections should 
be. The implementation reveals fast and effective trajectory learning. 

A major contribution! of this thesis is to demonstrate senna of the roles oF knowl¬ 
edge in learning. Powerful generalizations can be made on the basis of knowledge 
of system structure, as is demonstrated in the load and arm model estimation al¬ 
gorithms. Improving the performance of parameter estimation algorithms used sn 
learning involves knowledge of the measurement noise characteristics, as is shown 
in the derivation of optimal filters- Using trajectory errors to efficiently correct 
commands requires knowledge of how command errors are transformed into perfor¬ 
mance errors, le, an accurate model of the dynamics of the control ied system, as 
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is demonstrated In the trajectory learning wort. The performance demonstrated by 
the algorithms developed in this thesis should be compared with algorithms tiiat use 
less knowledge, such as table based schemes to learn arm dynamics, previous based 
trajectory learning algorithms, and much of traditional adaptive control. 

In order to evaluate the computational theory developed in this thesis learning 
algorithms have been developed and implemented in the context of robot arm con¬ 
trol. Although the experimental work in this thesis is restricted to work with robots, 
the insights gained have a much broader relevance. The adaptive mechanisms used 
to implement learning may differ In different domains, but at a computational level 
tile issues and principles remain the same. 


1.1 Choices and assumptions 

1.1.1 Working definition of learning 

The working definition of learning for the purposes of this thesis is improvement of 
pcrfoY itturtci with erperia nt.e.. Since the experimental work involves Ollty machines, 
motivational levels, development, and other factors that make analysing biological 
performance improvement difficult are not an issue. 

Two kinds of learning are studied. The first involves building and refining internal 
models of one’s self and the external world, and using those models to generate 
the appropriate actuator commands, and to guide processing of sensory data. The 
Second involves using internal models to transform performance errors into command 
corrections, 

1.1.2 Focus on arm trajectory control 

The effector system we wiil focus on will be a mechanical robot arm, and we will 
mainly be concerned with Improving execution of free arm trajectories, either with or 


without a load, The types of learning appropriate for this sort nf task are probably 
different front the types of learning involved in riding a bicycle or juggling. The 
ideas in this thesis will hopefully generalize and be useful in studying these other 
tasks, however. 

To provide a framework for what follows let US examine a typical robot control 
architecture. We can divide the robot control problem into deciding what to do 
(ptannt'ngr) and doing it [tieetdt'ott). For current robots planning Er usually accom¬ 
plished by a human programmer, although a goal of this and much other research is 
to incrementally automate robot programming. 

Since wc are concerned with robot trajectory execution, ei plan is a complete 
specification of the motion of the robot in some coordinate system. Often the plan 
is expressed in tosife coordinates, and in order to execute this plan we convert task 
coordinates to arm coordinates, This process is referred to as tncrflrse Jtmemafica. 

The next step in making the robot follow the desired trajectory is supplying 
appropriate commands to the actuators. The simplest approach to this is to generate 
these commands hy measuring the difference between where the arm is and where 
it iis supposed to be at any instant In time and using some function {usually a linear 
function) of this error as the drive signal to the actuators. This is what is known 
as feedback control. This type of contra! is useful in compensating for unpredicted 
disturbances. 

One problem with pure feedbsxk control is that it requires errors in order to 
generate any drive signals. Wo can avoid this problem by calculating our best guess 
as to the appropriate co mm and signals to apply to the actuators to follow the desired 
trajectory exactly. This precalculated command is added to the feedback command 
during execution of the motion. I will refer to applying a precalculated actuator 
command as. command feedforward control Ot simply feedforward control. The process 
of calculating the appropriate actuator command from a specification of the desired 
robot motion is referred to as rnirarje dynamics* 
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At this lave) of detail the same problems of coordinate transformation, predic¬ 
tion of appropriate actuator {muscle) commands, and compensating for nnpredicted 
disturbances must be addressed by biological systems. 

As a starting point for OUT discussion dF learn mg, let us ask the question, ^How 
might we go about building learning into tills performance architecture?" or equiv¬ 
alently (from the point of view of this thesis) ‘‘What information can be extracted 
from past experience to improve present and future robot performance? s 

Making better plans; Planning can be Improved in several waya: the. world 
models used In planning can be refined, better methods for solving the given task 
can be generated, and the planning methods themselves can be changed. These 
processes are assumed to be independent of improving execution of a given plan, 
and will not be addressed in what follows, W'e will take the plan as fixed. 

Improving execution of fixed plana; The thesis will focus On execution 
and making the effector system obey a given plan more closely. All, components 
of execution can be Improved using experience. One way of Improving the various 
coordinate transformations involved in robot control is to refine the kinematic model 
of the robot using measurements from redundant Bousing such as vision of the robot 
tip and joint angle sensing. To improve feedforward control we Could refine the 
dynamic model of the robot. We could also design a belter Feedback controller using 
the past history of controller actions. 

Assume kinematics is essentsally perfect: We will assume our kinematic 
model of the robot is essentially perfect, as there is already much research going on 
ill the area of kinematic calibration. 

Assume a fixed feedback controller: As there is currently much research in 
adaptive feedback control we will also take our feedback controller to be fixed, 

In a complete robot learning system plan learning, kinematic learning, adaptive 
feedback control, and the types oF Learning discussed in this thesis will all occur 
simultaneously. An important point to keep in mind is that they can be handled by 
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essentially independent modules. 

Focus on dynamics and control; Tit this thesis wc will discuss how to build 
and refine a model of robot dynamics to be used for predicting, the appropriate 
actuator commands to drive the robot (feedforward control). We will discuss how to 
handle certain types, of loads, Wc will also discuss how to build a model of noise in 
sensory data, and how to optimally filter that noise in the model building process. 
In addition we will ebaw the roie of the robot model as the learning operator during 
movement practice, ie, the robot mode! transforms, trajectory following errors into 
feedforward command correct ions, 

1..2 Outline of thesis 

1-2-1 Format of thesis 

Many of the chapters arc designed to be independent papers. The price of trying 
to write a thesis whose chapters are publishable Is some repetition and seeming 
multiplicity of goals, 

1.2.2 Introduction and Previous Work 

This chapter attempts to give the reader an idea of where he is going, why', and some 
of the Landmarks to look for. Chapter 2 t Previous Work, reviews some of the related 
research. Other related research is reviewed in the appropriate chapters. 

1.2.3 Arm and Laud Identification 

Chapters 3, 4, and 5 describe building models for one's own arm and any rigid 
body load. Appendix A describes the use of an identified model for the control of a 
particular robot arm. 
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This work demonstrates the power of knowing the system structure for teaming. 
In, this case the knowledge of the rigid body dynamics structure dramatically reduces 
the number of parameters to be learned and increases the learning rate compared 
to tabular Learning schemes (Aibus, 1975a, lS75b, Halbert, 1977, 197&, Raibert and 
Horn, 1978), More importantly, knowing the system structure allows much more 
powerful generalizations to he made. There are two constraints that can be used as 
sources of generalization. The first, is smoothness of the input/output transforma’ 
tlon. This constraint can be used to generalize between similar movements and is 
relied on heavily by tabular learning approaches. The constraint used in this work, 
system structure, la more powerful in. that information gained from one movement 
can be used to guide quite dissimilar movements. 

The crucial insight of these chapters is that systems dominated by rigid body 
dynamics can be described by a relatively small set of inertial parameters that appear 
linearly in the dynamics equations. Estimating these parameters turns out to he 
simple, as the theory and implementations show. We start with load identification 
[Chapter 3.) because that is the simplest version of the story, and with very little 
additional work we apply the same ideas to arm identification [Chapter 4). 

The reliance on system structure to achieve generalization does have a price,, 
however. The models do not represent well deviations from the assumed structure, 
which are always present to some degree. Thus, on any particular trajectory ex¬ 
ecution a rigid body dynamics model will have small errors. Charging the model 
parameters to fit this trajectory more exactly will degrade performance on other 
trajectories. What ts required is an additional level of modelling that allows repre' 
aentation of fine details of the dynamics. This correction model and how it is learned 
is described in Chapter 6, Single Trajectory Learning, 

Chapter 5, Optimal Filtering For Parameter- Estimation, discusses the practi¬ 
cal issues of handling noisy measurements. How to optimally process the incoming 
sensory data is determined by models of the measurement noise. Improving the pet- 
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formance of parameter estimation algorithms used in learning involves knowledge oF 
the measurement noija: characteristics, which are estimated from redundant sensing. 
Thus, the Learning elements themselves [earn, in this ease by building better models 
of the sensors and their interaction with the world, 

Aii Insight that simplifies system identification is to use sensing to decouple dif¬ 
ferent system identification problems. We use a wrist force/torque sensor that might 
allow us to estimate load parameters independently of the arm dynamics, for exam¬ 
ple. More general Ey, covering an arm or any effector system with a sensory barrier 
such as a tactile sensing skin allows ns to separate identifying internal system dynam¬ 
ics and identifying external system dynamics and disturbances. Without a sensory 
barrier the combined system of arm plus external world may be too complex to 
identify robustly. Decoupling system identification problems allows different identi¬ 
fication procedures to be applied to the different subsystems. In the case of an arm 
with variable loads T we expect the arm dynamics to be only slowly varying, while 
Load dynamics change rapidly as loads are picked up or put down. Widely different 
rates and types of system change call for different system identification algorithms 
to track the changes. Another reason for decoupling arm and load identification is 
that the arm structure is relatively constant while the dynamics of different loads 
can have quite dlffcrant structures. Arm identification can be baaed on a fixed model 
structure, while a complete load identification system must handle many different 
model structures. 

It Is important to keep in mind the broad relevance of the model building ap¬ 
proach suggested here. It is true that other sources of dynamics 81 ich as friction 
and actuator dynamics play an important role in the dynamics oF many current 
robot arms. However, l) rigid body dynamics is often a source of coupling dynam¬ 
ics between the joints, 2) Since friction and actuator dynamics are usually isolated 
aL particular joints their identification reduces to a single input single output mod¬ 
elling problem rather than the more difficult multiple input multiple output problem 


solved here, and 3) the same process of hypothesising a mode] structure and esti¬ 
mating parameters can be used to identify Friction and actuator dynamics and can 
even be added to the linear parameter estimation algorithms used in this work, 

1.2.4 Single Trajectory Learning 

Chapter fi, Single Trajectory Learning, addresses the issue of using practice to im¬ 
prove the execution of a particular trajectory. The crucial insight of this chapter is 
that one roust use an accurate model of the controlled system to make sense of trajec¬ 
tory errors, ie, convert the errors, into corrections to feedforward commands With¬ 
out an accurate model attempts to Improve trajectory performance will most likely 
degrade performance. This work demonstrates the role of knowledge in. analysing 
past behavior and correcting previous mistakes, and should be compared to the 
proliferation of ad hoc trajectory Learning schemes. 

We are able to mathematically analyse the effect of various proposed single tra¬ 
jectory learning algorithms, which tells us why one Learning operator works belter 
than another. The most Important result is that the convergence rates of the algo¬ 
rithms are determined by the quality of the models used. We can put mathematical 
bounds on acceptable modelling error for the linear case, 

The model used in the trajectory learning work is the identified arm model pre¬ 
sented in Chapter 4, Thus, starting with only knowledge of system structure, wo 
have demonstrated a system that cau build a general model of itself after only three 
or four movement A, and then can learn to execute any particular trajectory to almost 
the limits of the system repeatability in an additional three or four movements, 

1.2.5 Future resualreh 

The final chapter discusses what to do next. This work suggests a program of 
research on the quantitative psychophysics oF motor learning. TLigid body load iden- 


14 


tificAtion is a first step towards general force perception, The extension of the 
trajectory learning algorithm to improve performance on A group oF similar trajecto¬ 
ries Is discussed. Also included IS a discussion of performance simplification, which 
reduces the need for detailed internal dynamic models by restricting possible motor 
performance. 
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Chapter 2 


Previous Work 


This chap ter spells out the theoretical backgrou nd and philosophy of our approach, 
and surveys the relevant engineering literalure. This thesis address™ the following 
questions; 

1. How can we generate appropriate actuator command signals to drive a con¬ 
trolled system along some desired trajectory? 

2, How can we modify the actuator command signals to improve trajectory execu¬ 
tion on the same and similar trajectories, given data from previous, movements? 

This chapter describes the genera] approach taken and contrasts it with approaches 
taken by others. In summary* we 

1 . Describe three basic types of controllers; 

(a) Feedback 

{b) Disturbance Feedforward 
(c) Command Feedforward 

and their roles, 


in 


2 , Describe why we have chosen to pursue command feedforward control t and 
specifically adaptive command feedforward control, and 

5, Describe previous work on adaptive command feedforward control. 


2.1 Introduction: Three types of control and adap¬ 
tive control 

The motivating control problem: The areas of control theory that one is in¬ 
terested in are often shaped by the systems one desires to control. The emphasis 
here is on the control of biological and robotic arms during the execution of free 
trajectories, 

Focusing on execution; We are taking a modular approach to motor learning, 
and have split learning into two parts: making better plans and following plans mure 
closely. In this chapter we assume there Is a given plan or desired trajectory and our 
goal is to execute it an accurately as possible, Thug we focus on the controller, the 
element which transforms plans Into actuator commands, and how we might refine 
the controller with experience. Following plans more closely ie the goal of adaptive 
control, the branch of control theory that addresses how to improve controllers based 
oil experience. 

Three typos of control? A control system can be divided into three parts: 
the feedback controller and two feedforward controllers (Ogata, 1070; Takahasht, 
Rabins, and A us lander, 1070; Astromand WEttcnmark, 19&4). Figure 2,1 shows; the 
general form of such a system,. 

Feedback control is any control action based on the actual state history of the 
controlled system, and feedforward control is all other control actions (D’Arzo and 
Iloupts, 19GG; Astrom and Wittenmark, 13^4). We can distinguish two types of 
feedforward control: feedforward commands ma 3 r be based on measurements of Other 


17 



Figure 2.1: The three parts of a. control system 
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variable* that affect the controlled system but arc not affected by it (disturbance 
feedforward) or the commands may be based on predictions of the control signals 
necessary to drive the controlled system along some desired trajectory (command 
feedforward). 

Three typos of adaptive control: Corresponding to the different parts of the 
controller there are different types of adaptive control. Adaptive feedback control 
involves adjusting the feedback controller. Adaptive feedforward control may Involve 
adjusting gains from measured signals (adaptive disturbance feedforward control) or 
improving the models used for control signal prediction (adaptive command feedfor¬ 
ward control). We will focus on improving the models used for command feedforward 
control. 


2.1-1 Questions that have come up 

Generality of approach: Each component of the control system (plant, feed¬ 
back controller* disturbance feedforward, command feed Forward) may be a complex 
non-linear transformation. We will initially use simple examples (typically linear 
systems) to present concepts clearly, but the reader should keep ill mind that much 
of what is said applies to more general systems- 

B,oference arid feedforward signals: Note that there are many equivalent 
ways to think about the command feedforward part of the control system, especially 
if the system components arc linear (Horowitz, 19G3), We can think of a reference 
signal to the feedback controller as another feedforward signal. For conceptual clar¬ 
ity, we want to separate modifiable feedforward actuator commands from reference 
signals to the Feedback controller. The reference signals will always be the desired 
trajectory. 

Terminology: Although the control terminology is not completely standardized 
the term “feedForward* is used quite often in the literature; for example, feedfar- 
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ward control! for measured diEturbanccs. ( Grib be, Ramo, and Wooldridge, 1961; 
Peschon, 19&5* Savas, 1CH35; Weyridt* 1975; Marshall* I97B; Owens, 1978; A rand, 
19 S4 - SchwarEenb&ch and Gill* 1984; Leigh* 1985 )* feedforward contra] for com¬ 
mand inputs [ Ton, 1059; Webb, 1964; Eveleigh, 1072 )* feedforward control for 
disturbances and commands ( Macmillan, 1051; Doebdin, 1962; Shinskey* 1963; 
Wool vert™ and MurrilL* 1997), In the Russian literature the "Theory of Invari- 
aricc B is often invoked to show how and why feedforward control should be used to 
make the system error independent of commands and disturbances. This has also 
been referred to as Toncdet^s principle 1 ” [Preminger and Rootenberg* 1964). Com¬ 
mand Feedforward Is often referred to as an input or reference prefilter* especially if 
this signaE is used as the reference to the feedback controller. Combi ned feedback 
and feedforward control has been referred to as a “Hwo degree of freedom control 
system." (Horowitz, 1963). There is occasionally some confusion in the use af the 
term feedforward. ^Feedforward controller" has been used to refer to any element, 
in the forward path of the closed loop controller (Raven, 1978; Saridis* 1977). These 
dements are clearly part of the feedback controller. 


2.2 The Feedback Controller 

Feedback control is the only way to handle unmeasured or unpreditted disturbances* 
modelling errors, and to stabilise unstable plants, and thus plays a critical role in 
control in general and in robotics in particular. Much effort is going into robot, 
feedback controller design, sometimes to the exclusion of considering other types or 
control. However* there are limits to what can he done with feedback control* and 
feedforward control can usefully augment a feedback contra] ler, It should be noted 
that the feedback controller can be designed to a large extent independently of any 
feed Forward control* separating issues qf robustness to disturbances and modelling 
errors, and command tracking [Horowitz, 1963), 


2*2-1 Limits on feedback controller 


Control of terminal compliance or, mare generally, impedance haa been proposed as 
a goal for robotic contra!. Such control may require limiting feedback gains, If the 
desired compliance is implemented as a tow gain position servo, For nan-redundant 
robots with no terminal force/torqtie sensing choosing an impedance specifies the 
feedback controller completely. 

The use of a force sensors at the interface between the robot and its Jo-ad or 
environment may allow differential rejection of modelling errors and external fortes. 
In practice, however, this hag not yet been shown to work well. 

In biological systems there are additional limits on the possible feedback gains 
due to transmission delays. In general, non minimum phase elements such as delays 
(which also occur in machines) and right half plane Eeros set limits on maximum 
feedback gains as do modelling error, ignorance of plant dynamics, parameter vari¬ 
ations, and measurement, or command noise. (Astrom and Wi tten mark, 11)84) 

2.2.2 Plenty of work going on in adaptive feedback control 

Approaches to adaptive Control have been almost exclusively focused on modifying 
the feedback controller (W. D, T. Davies, 1970, Tsypkin, 1971 j Witt enmar k. 1975; 
Saridis, 1977;; Landau, 1979; Harris and Billings, 1981; Isermann, 1982; Astrom, 
1983; Landau, Tomizuka, and Auslaudcr, 1933; Astrcun and Wittenmark, 19-34; 
Goodwin and Sin, 1984; Voronov and Rutkoveky, 1984; Yale, 1985). 

There Etas also been great interest in adaptive feedback control in robotics. The 
added complexity of an adaptive controller is typically justified on the basis of the 
complexity of the robot dynamics, the belief that the parameters in robot models are 
difficult to estimate, that loads wiJL be unknown, and that there will be significant 
modelling errors caused by friction (static and dynamic), joint compliance, actuator 
modelling errors, and link flexibility, Many of the justifications for adaptive con- 
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trol arc not valid! and it Ls not clear that adaptive feedback controllers have any 
advantage over well designed non-adaptive controllers, Furthermore, the limitations 
on feedback control discussed previously limit the performance «F adaptive feedback 
controllers just aa they limit fixed gain controller performance. 


2,3 Feedforward Control 

Feedforward control is any control action that is independent oF the previous actual 
state history of the plant, and is used to improve command following and counteract 
predictable or measurable disturbances. Feedforward control lias been used for a 
long time (Graham, 1916; Moore, 1951; Preminger and R.ontenberg, 1963) + and in 
fact many control systems use feedforward control exclusively and thus are pure 
open loop controllers, 

AeriMjmc-e applications i Feedforward control i& used extensively in aerospace 
applications and is related to the use oF optimal control there, Typically a rocket 
trajectory 5a planned according to some cost criterion and the thrust history neces¬ 
sarily to follow that trajectory is pre-ealcuiated, The feedback controller counteracts 
deviations from the planned trajectory by modifying the nominal thrusts. Adaptive 
feedforward control has not been emphasised In these applications aa the same rocket 
is rarely used twice. 

Process control applications: Feedforward control has also been used in pro¬ 
cess control systems (Macmillan, 1951; Shinskey, 1963; Sav&s, 1965; Weyrick, 1975; 
Marshall, 1970; Shinskey, 1979), These applications tend to focus on counteracting 
measured disturbances since desired outputs tend to be constant for long periods dF 
time and outputs during transients or changes in production are usually discarded. 
Also, process control fa characterised by long delays, large time constants [slow 
processes), and measurable disturbances. 
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Robotics applications! Feed Forward contra) is an important part of robot 
control (Orin et aL, l'ST'&j Liegeois, FpflMH, and Aldou, 1990; Paul, I9Bi; Brady 
et &L, 199-2; Asad a, Kanade, and T&kyuna, 1953; Gcrstenbergcr, 1965; Hollars and 
Camion, 198o; Liegcois, L985), although there is some variation in the terminology; 
“Inverse problem” (Pan], 1972); "Computed torque” (Markicwitss, L973; Bejczy, 
1974); “Two itage 11 control (Vukobratavic and Patkanjak, l9ft£; Yukobrattwii;. and 
Stokic, 1982). 

There are several reasons for the emphasis on feedforward control in robotics: 
I) Often the task or movement trajectory is specified in advance, and 2) the forces 
necessary to drive the robot along the specified trajectory are usually dominated by 
the forces necessary to drive the robot itself, and not the load, and thus are to a 
large extent predictable. 3) Most robots must Interact compliantly (l.e. with low 
stiffness) with the environment and thus feedback gains are limited. 4) OFten the 
task or trajectory is repetitive and the disturbances are the Earns on each execu¬ 
tion. 5) Computers are typically available for the generation of complex feedforward 
commands. The goal of robot trajectory control is to minimize deviations from an 
arbitrary desired trajectory, which with a limited gain feedback controller requires 
accurate models for the generation of feedforward commands and the ability to refine 
those feed Forward commands on rep eated executions of the same trajectory. 

Biological applications: Biologists have used the concept of feedforward con¬ 
trol to tinders Land commands based on measured signals an-d commands based op 
predicted forces (Szentigothai and Arblb, 1975). The vestibular ocular reflex is 
an example of both disturbance feedforward control lend adaptive disturbance Feed¬ 
forward control, Fast limb movement is hypothesised to he driven by command 
feedforward control, as the delays in feedback control in biological systems limit 
possible gains and feedback effectiveness. 

Technological limitations: To a certain extent the uSc of feedforward control 
has been limited in the past by the available technology' (Peschon, 1965; Savas, 


19G5; Isermaun, 19B1; Tsermann, Astrom and Wittenmark, 19S4). The cost of 
analog electrical, mechanical, pneumatic, fluidic, or hydraulic controller circuits is 
proportional to their complexity, and implementing derivative operator, nanliuear 
operations, time delays, or functions such as sin(^) is relatively difficult. These 
circuits are restricted to be causal processors, ana thus derivatives oF Lhe command 
must either he provided to the controller or approximated [An interesting mechanical 
differentiator is presented in Do-ebeltn, 1962, p,2SC), These derivative? are typically 
useful in feedforward control, Furthermore, the reliability of analog circuitry is 
inversely proportional to its complexity, and this has encouraged the use of simple 
controllers. Some mechanism for information storage and retrieval is useful For 
feedforward control, as WC shall later demonstrate, and in the past this was restricted 
to such technology as mechanical cam following. The introduction, of computer 
control and the dramatic drop in the cost of computer hardware has greatly expanded 
the range of possible controller designs. 

2.3*1 Feedforward Controller Design 

It is well known that the ideal feedforward controller incorporates a model of the in* 
veree. of the plant [Graham, 1946; Moore, L&5I; Tou, 1959; Shtuskcy, 1063; Evelcigh, 
1972; Weyrick, 1975; Owens, 19=78; Shinskey, 1979; Isermann,. 1931; Isermann, 1952: 
Astrom and Wittenmaik, 19&4), One implication of this is that adaptive Feedforward 
control involves building better models of the plant. 

The need to invert the plant immediately raises the question of what to do when 
the inverse does not exist or is not realizable. This occurs when the plant has any 
of the following features: 

I. Time delays: The inverse of a time delay is a time advance:, which is not 
realisable, 

2 r Right half plane zeros: A causal inverse of a right half plane zero is unstable, 
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and a stable inverse is not realizable since it requires control actions In the 
infinite past, 

3, Many to one mappings; A plant may map many sets of inputs into the 
same outputs, and thus formally the plant inverse does not exist. This is not 
typically a real problem, as additional constraints can be added to resolve the 
many to one mapping, or a partite lar inverse can be chosen, 

4, Singularities: Points or sets of points at which a nonlinear or time varying 
linear plant mapping changes dimension may complicate inverting the plant. 
We will rely on the planning stage to avoid this problem, at least for now. 

We see that realizability of the plant inverse is a critical issue in feedforward 
command design. This leads us to distinguish disturbance feedforward and command 
feedforward. 

L Future commands are Hqnnwnt A disturbance is assumed to be unpre¬ 
dictable, or at least to liave an unpredictable component „ while we assume that 
commands are known as far into the future as necessary. Of course, the fu¬ 
ture command may change, and this must he accommodated by the command 
feedforward controller. Some amount of acausality in command feedforward 
generation is acceptable. The disturbance feedforward controller, however, 
must be a causal proceseor- 

2 . Commands have negligible noise: In adds Lion, disturbance measurements 
may be corrupted by noise, while commands, being internal to the controller, 
are assumed to be have negligible noise. Thus, derivative operators. can accu¬ 
rately take derivatives of the command. 

Tbe goal of disturbance feedforward controller design is to design & causal and 
stable system whose input is disturbance measure merits and whose output is ap- 
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proptiatc control actions, Approaches to designing such a system are discussed in 
(Isermann, 19S1), 

It Is possible to treat ea.rrmna.nd feedforward contra] as a type of disturbance 
feedforward control and attempt to design a causal command filter. This is not 
necessary the hast approach, however. The goal of command feedforward control is 
to design the particular actuator command signal for a particular desired trajectory. 
This processing need not be causal, To bundle & plant delay it is possible simply to 
shift the command backwards in time. To handle a right half plane zero it is possible 
to design commands that do not grow to infinity. This typically requires that the 
command starts in the infinite past. Since this is clearly not possible this type of 
command must be truncated, leading to unavoidable trajectory following errors. 


2-4 Previous Approaches To Adaptive Feedforward 
Control 

2,4.1 Self Tuning Adaptive Feedforward Control 

A straightforward approach to adaptive feedforward control is to treat command 
and disturbance feedforward control identically and simply automate existing feed¬ 
forward controller design procedures, initially the plant and any disturbance effects 
must be Identified, ami then a controller is designed from that. There are typi¬ 
cally two steps to identification: choosing a model structure and then estimating the 
free parameters in that structure. By making different choices in mode] structures, 
parameter estimation procedures, and feedforward controller design many different 
adaptive feedforward control schemes can be derived. This approach IS analogous to 
“Self Tuning" adaptive feedback control (Isermann, 1981’ Isermann,. 1982; Astrom, 
19&3; Astrom and Wittenmark, 1981), The perils of right half plane zeros are typi¬ 
cally avoided by using a model structure that cannot contain right half plane seros. 
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This may lead to an approximation of the true piant + but the inverse of the mode] 
will always be stable. 

Woolverton and Murrill (1967) suggest choosing initial parameters for a second 
order feedforward controller from, experiments on the control led process and then 
a trial and error exploration of the nearby parameter space to find the optimum 
parameter set. 

Bristol (1967) chooses a simple linear model structure for a heat exchanger, 
V = a ‘L + fr, and closes an integrating feedback loop around the free parameters a 
and £ to continuously update this model. The model is concurrently used to generate 
feedforward commands, V n from load measurements, L, An objective of this Scheme 
is to only include operations easily implcmentiubla in analog hardware and the final 
controller was pneumatic. 

Bernard and Lefkowiti [1962) chose a more complicated, process model 

Kc" Dt 

= im <**> 

where P(e) is the process transfer function, K is the gain,. D is the delay, and L 
is a time constant. Initially K, D, and L are estimated from previously collected 
data, and then D and L arc continuously updated. Tills, model is used to generate 
bang-hang control responses to step changes in measured disturbances, 

Wittcnm&rk (1973) derives a selF-tuning minimum variance feedforward con¬ 
troller, 

Schumann and Christ [1979) present the most extensive exploration of self timing 
adaptive feedforward controllers. They make explicit three steps; 

1. Choose structural parameters of process and disturbance models; model order, 
time delays, sampling interval, etc, 

2. Use some recursive parameter estimation scheme to identify the model param¬ 
eters. Typically recursive least squares is used although if there is significant 
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process noise an unbiased scheme such as recursive maximum likelihood is 
mere appropriate. 

3. Design a feedforward controller based on. the identified, model. Here we assume 
the certainty equivalence principle ia valid. 

The process and disturbances are modelled by separate ARMA mo-deie. Since their 
model structure allowed right half plane zeros they needed feedforward design pro¬ 
cedures that handled this problem, They explored seven feedforward control design 
procedures, 

Elicabe and Moira (]i9£3j extend the adaptive model following control approach 
(Landau, 1979) to adapt a feedforward controller to handle measurable disturbances 
Wldrow and colleagues [Widrow, Walach, and Shaffer, 1993-; Widrow and Walach, 
19S3^ W id row et a|,, 1991; Anderson and Johnstone, 19S1; Widrow, McCooll, and 
Medoff, 1970] model the plant as an all pole process, thus avoiding the right half 
plane zero problem. They use an LMS (leaat mean square} algorithm, to continuously 
model the plant. They approximate the true plant In a least squares sense, and arc 
able to drive the plant approximatsEy along a trajectory after a certain delay. 


Adaptive feedforward control in robotics 

2.4.2 Adaptive Global Models 

One approach to adaptive command feedforward control is to attempt to identify a 
model of the robot dynamics, and invert the model, or to directly identify an inverse 
model of the plant. In this section we will outline approaches which attempt to 
model the robot dynamics under all conditions with one model. We refer to this 
approach as the globaii modelling approach. 

One approach for such global modeling is to derive a rigid body dynamics model 
of the arm. and identify the parameters of that model. Examples of such work are 


discussed in Chapter A and include Mayeda, Qsuka, and Kangawa (19R i} T Mukerjec 
(J9EH), Mufcerjee and Ballard (1985), Olsen and Bekey (1985), Neuman and KJiosla 
[1Q8&1, Abl s Alkcsurs, and Holler bach (1985), Kbosla and Kanade (1&95). 

Goor (1985a, 1985b) emphasized the actuator dynamics and claimed a decoupled 
third order model of each joint of the robot was more appropriate. He incorporated 
a third order command feed Forward filter and developed an algorithm that esti¬ 
mated the coefficients of that filter tn real lime* He also emphasised the necessity 
of commanded trajectories to he executable, i.e. not require infinite inputs. 

A Stay issue in adaptive global modeling is that the identified model should be 
constant or at least independent of the state of the plant, even during large motions 
of the robot. If this is not the case then it is not clear that there is enough data 
during the movement of the robot to identify a new model at each significantly 
different state. 

Another key issue is to insure that the data used to estimate the robot model 
cover & wide range of states and inputs. Otherwise structural modelling errors 
may drive the parameter estimation procedure: to seriously distort the estimated 
parameters to fit a small range of data. This can cause predictions for other ranges 
of data not used In the estimation to deteriorate, and the advantage of global models, 
generalization, is lost. 

2.4.3 Lookup Table Based Approaches: Local models 

Lookup table based approaches have been suggested in the past as a solution to the 
model evaluation problem: given an existing model can we compute the necessary 
inputs- to achieve the desired outputs fast enough? [Albius, 1.975a,, 1975b; ftaibert, 
19TT, I97B; R.aibert and Horn, 1978) One way to fill the tables Is using some form of 
learning. Tables do provide an easily modifiable representation of a transformation. 
However, a straightforward implementation leads to huge table sizes, and consider - 
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able effort must be expended En designing how data is actually stored, and how It is 
put in and taken out of the table. A major Elaw of tabular approaches is that it is 
difficult to take advantage of the structure of the arm dynamics,, bo much more, data 
has to be collected to learn to the same level of performance as the algorithms de¬ 
scribed in this thesis, and generalisations between dissimilar movements are difficult 
to make, 

We will eventually be suggesting table based approaches to form a middle level 
of models: local models. Global models help us achieve generalisation, and single 
trajectory models allow us to follow a single trajec tory with zero error after practice. 
Lookup table based local models can help us modify commands for a particular 
trajectory to achieve similar trajectories. An important distinction between our 
approach and previous approaches is that we propose using a hierarchy of several 
different types of models to gain the benefits of each and the drawbacks of none, 

2.4.4 Iterative Trajectory Learning: Single trajectory mod¬ 
els 

At the Opposite extreme of global modeling is the modeling of the feedforward com¬ 
mand for ft single trajectory. Through repeated attempts to follow a single specified 
trajectory the feedforward command is refined, and ultimately the trajectory is fol¬ 
lowed with zero error. The key issues here are the stability and convergence rate of 
the iterative process, and how to design the learning operator. One drawback of this 
approach is that it only produces the appropriate command for a single trajectory. 
There is little guidance as to how to modify that command for similar trajectories. 
Tliis approach le discuBEed in Chapter G. Single Trajectory Learning, 


Chapter 3 


Rigid Body Load Inertial 
Parameter Estimation 


3,1 Abstract 

A method for estimating the mass., the renter of mass, and the momenta of inertia of 
a rigid body load during general manipulator movement is presented. The algorithm 
is derived from Hue NeWtOfi-Euler equations and incorporates measurements oF the 
force and torque from a wrist force-torque sensor and of the arm kinematics. The 
ider: Li Flea Lion equations are linear in the desired unknown parameters, which are 
estimated by least squares. We have implemented this identification procedure on a 
PUMA 600 robot equipped with an RTI FS-B wrist force-torque sensor, and on the 
MIT Serial 7unk Direct Drive Arm equipped with a Larry Wright Company Astek 
wrist force-torque sensor. -Good estimates were obtained for load mass and center 
of mass, and the forces and torques due to movement of the load could he predicted 
accurately. The load moments of inertia were more difficult to estimate. 

1 Thi» dhapler is fl. revised version tj l] A lie-, son. Am f and ETo].i!rl»iLi:]i, ItflSa.) 
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3.2 Introduction 


This chapter presents a method of estimating all of the inertial parameters of a rigid 
body load using a wrist Force-torque sensor: the mass, the moments of inertia, the 
location of Its center of mass, and the object's orientation relative to a force sensing 
coordinate system- The procedure has three steps: 

1, A Newton-Euler formulation for the rigid body bad yields dynamics equations 
Linear in the unknown inertia!! parameters, when the moment of inertia tensor 
La expressed about the wrist sensing force coordinate system origin, 

2, These inertial parameters are then estimated using a least squares estimation 
algorithm, 

3, The location of the load’s center of mass, its orientation, and its principal 
moments of inertia can be recovered from the sensor referenced estimated 
parameters, 

In principle, there are no restrictions on the movements used to dt> this load identifi¬ 
cation, except that if accurate estimation of at] the parameters is desired the motion 
must be sufficiently rich (ie,, occupy more than one orientation with respect to grav¬ 
ity and contain angular accelerations in several different directions). In practice, 
however, special test movements must sometimes be used to get accurate estimates 
of moment of inertia paraineters. 

There are several applications for this bad identification procedure. The accu¬ 
racy of path following and general control quality of manipulators moving external 
loads can be improved by incorporating a model of the bad into the controller, aa 
the effective inertial parameters of the last link of the manipulator change With the 
load- The mass, the center of mass, and the moments of inertia constitute a com¬ 
plete set of inertial parameters for an object; in most cases, these parameters form a 
good description of the object, although they do not uniquely define it, ThE object 
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may be completely unknown at first and an Inertial description of the object may be 
generated as the robot picks up and moves t.hc object. The robot may also be used 
in a verification process, in which the desired specification of the object is known and 
the manipulator examines the object to verify if it is within the tolerances, Recog¬ 
nition, finding the best match of a manipulated object to one among a, set of known 
objects, may also- be -desired. Finally, the estimated location of the center of mass 
and the orientation of the principal axis can be used to verify tliat the manipulator 
has grasped the object in the desired manner, 

A key feature of our approach is that it requires no special test or identification 
movements and therefore can continuously interpret wrist force and torque sensory 
data during any desired manipulation. Previous methods of load identification were 
restricted in their application. Paul [1901] described two methods of determining 
the mass oF a Load when the manipulator is at rest, one requiring th« knowledge of 
joint torques and the other forces and torques at the wrist. The center of mass and 
the load moments of inertia wore not identified, 

CoifFet (19S3) utilised joint torque sensing to estimate the mass and center of 
mass of a load for a robot at rest. Moments of Inertia were estimated with special 
teat motions, moving onty one axis at a time or applying test torques. Because of 
the intervening link masses and domination of inertia by the mass moments* joint 
torque sensing is Leas accurate than wrist force-torque sensing, 

Olsen and Befcey (L985) assumed full forte-torque sensing at the wrist to identify 
the toad without special test motions. Mukerjee (1984), and Mukcrjec fit Ballard 
[1FHS5-) developed an approach similar to ours, again allowing general motion during 
load identification. Nevertheless, neither paper simulated Or experimentally imple-. 
merited their procedures to verify the correctness of the equations or to determine 
the accuracy of estimation in the presence of noise and imperfect measurements. 

Our algorithm requires measurements of the force and tuque due to a load and 
measurements or estimates oF the position, velocity, acceleration,. orientation, angu- 
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lar velocity'., and. angular acceleration of the force sensing coordinate system. The 
algorithm can handle Incomplete force and torque measurement by simply el I mi- 
nating the equations containing missing measurements. The necessary kinematic 
data can be obtained from the joint angles and, if available,, the joint velocities of 
the manipulator. The inertia] paraEneters of a robot hand can be identified using, 
this algorithm and then the predicted force* and torques due to the hand can be 
subtracted from the sensed forces and torques. 

This inertial parameter estimation algorithm was implemented using a. PUMA 
600 robot equipped with an RTI FS-B wrist force-torque sensor, and on the MIT 
Serial Link Direct Drive Arm (DDA) equipped with a Barry Wright Company Astek 
FS6-12DA'2'00 6-axis wrist force-torque sensor. 


3.3 The Newton-Euler Approach To The Load 
Identification Problem 

3.3*1 Deriving The Parameter Equation 

To derive equations for estimating the unknown inertial parameter, the coordinate 
systems 3u Figure 3.1 are used to relate different coordinate fruueg and vectors. Q 
is an inertial or base coordinate system, which Is fixed in space with gravity painting 
along the —a axis. P is the force reference coordinate system of a wrist force-torque 
sensor rigidly attached to the load. Q represents the principal axis of the rigid body 
load located at the center of mass. The x axis of Q is along the largest principal 
moment of inertia, and the g axis along the smallest, Q is unique up to a reflection 
in bodies with 3 distinct principal momenta of inertia. In the derivation that follows 
ail vectors are initially expressed in the base coordinate system D. 

The mass, moments of inertia, location of the center of mass, and orientation of 
the body (a rotation , r H from the principal axes to the force reference system) arc 
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FORCE REFERENCE FRAME 



p; position vector from the origin of the base coordinate frame 
to the origin of the wrist sensor coordinate frame, 

tji position vector from the origin of the base coordinate frame 
to the center of the mass of the load, 

c: position vector from the origin of the wrist sensor coordinate 
frame to the center of the mass of the load, 

Figure 3.1: Coordinate Frames. 
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related to the motion of the load and the Forces and torques exerted on it by the 
Newton-Euler equations. The net force and the net torque ? u acting on the load 
at the canter of mags are: 

f f = f + mg = rn q (3.1) 

fix^n-cxf^Ew+vx (3,3) 

where: 

f — the force exerted by the wrist sensor on the load at the 
point p, 

in — the mass of the load, 

g = the gravity vector [g = |0,0,-SLSrneters/sec 1 ]},, 

Q = the acceleration of the tenter of maw of the load, 
n — the torque exerted hy the wrist sensor on the load at the 
point p, 

c - the unknown location of the center of mags relative to the 
force sensing wriat origin P t 

jl — the moment of inertia tensor about the center of mass, 
tit = the angular velocity vector, and 
6j = the angular acceleration vector.. 

We need to express the force and torque measured by the wriat senaor in terms of 
the product of known geometric parameters and the unknown inertial parameters. 
Although the location of the center of mass and hence its acceleration q are unknown,, 
q is related to the the acceleration of the force reference frame p by 

q-p+ijxc + uxfwxc) [3 [7,40] s ) 

Substituting (3) into (3.l) 5 

f — mp- mg + w x me + w x (tux me) (3,4) 

2 Equation uitinbem in braciketi refer to eqotsoBa. La Symoq, 18T1, 


36 



Substituting (3.4) Into [5,2), 

li — fl ItJ + uX (,r«) + me X (w X q) 4- me X (w X [w X c)) 

4- m? x p — me X % (5) 

Although the terms rr X (tb X q) and q X (w X (W X cj) are quadratic in the unknown 
location of the center of mass c, these quadratic terms are eliminated by expressing 
the moment of inertia tensor about the forcE sensor coordinate origin instead of 
about the center of mass j-I). Rewriting (5} as: 

H — jldf + UI X 4- - (qq T )]w 

+ w x {m[(c 7 c)l ■— (cc^) ttf} + me x jj — me x ^ (6) 

and using the three dimensional version of the parallel AXIS theorem 

,1 = ? I 4 mf{c f c]l - (ec T )] [7 [10.147]). 

to simplify it results in: 


n = plej 4- w x (jlw) + me x p - me x g 


m 


[1 is the 3 dimensional identity matrix). We now express All the vectors ill the 
wrist sensor coordinate system P t since then the quantities c and arc constant. 
Moreover, the wrist force-torque sensor measures forces and torques directly in the 
P coordinate frame. 


In order to formulate the above equations as a system of linear equations, the 
following notation is used: 


W X € = 


0 -w x w r 
0 — u x 

—w r w t 0 


c x 



Cx 


(3.9J 
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where 


Iw — 


w t w tf w, 0 0 0 

0 ll.'j 0 w, u, 0 


0 0 0 u a 


hi 


Ai 

hi 


As 

hi 

^ r 1 

As 


= [*w] 


ht 


As 

As 


As 

As 


. As j 


Ai 

A2 

As 

Ai 

As 

As 

As 

As 

As 


(3.10] 


[3.II] 


Using these expressions, Eqs, (3.4] and (3.0.) can he written as a single matrix 


equation in the wrist sensor coordinate frame: 


‘ f. 
A 
A 

n c 

"i 


p - K fwx] + [wx][wxj 0 

0 |(*“P)X] [au) + |ux][»u] 


Dr more compactly, 


w - A$ 


trtCj 

me, 

me, 

hi 

J 12 

hi 

hi 

hi 

hi 


[3.12] 


(3,13) 


■where w Ls a 0 element wrench vector combining both the force and torque vectors, 
A is a 6 x 10 matrix, and & h the vector of the 10 unknown inertia] parameters. 
Note that the center of mass cannot be determined directly, but only as the mass 
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moment -me. But since the mass m ia separately determined, its contribution can 
be factored from the mass moment later. 


3*3*2 Estimating The Parameters 

The quantities inside the A matrix are computed by direct kinematics computation 
(Luh, Walter, and Paul,, IS&Ob) from the measured joint angles, The elements of 
the W vector are measured directly by the Wrist force sensor. Since (3.13) represents 
fi equations and 10 unknowns., at least two data points arc necessary' to solve for the 
<f> vector* i.e. the force and the position data -sampled at two different configurations 
of the manipulator. For robust estimates in the presence of noise, we actually need 
to U£e a Sanger number of data points. Each data point adds fi more equations, while 
the number of unknowns, the elements of <£* remain constant, w and A can bo 
augmented with n data points: 


A = 

' A|l| ' 

» w = 

"HI' 


. A M . 


w[ft| _ 


where each A[ij and w[i] are matrix and vector quantities described in (3,12), For¬ 
mulated this way, any linear estimation algorithm can be used to identify the ^ 
vector. A simple and popular method is the Least squares method. The estimate for 
4> is given by: 

£= (A r A)-'A T w (3.1b] 

Equation {3,15} can also he formulated in a recursive form [Ljung and Soderstrom, 
1983) for on-line estimation. 

3*3+3 Recovering Object And Grip Parameters 

The estimated inertia] parameters [ra, me, jJ) are adequate for control, but for 
object recognition and verification we also require the principal momentR of inertia 
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J|, /?, is, the location of the center of man c, and the orientation ?F .R of Q with 
respect to P„ 

The parallel axis theorem is used to compute the inertia terms translated to the 
center of mast of the load. 



(3.16) 


,1 = ,i - *[(6 r «]l - («S T )] 


(3.1J) 


The principal moments are obtained by diagonalizing 



7 I — qpR- 


/, 0 0 

o !j o 

0 0 h 


(?P 


r t 


(3 18 ) 


This dEagonalization can, always he achieved since f I is symmetric, but when two or 
more principal moments are equal the rotation matrix, q^R, ia no longer unique. 


3*4 Experimental Results 

3.4,1 Estimation on the PUMA Robot 

The inertial parameter estimation algorithm was originally implemented on a PUMA 
f>00 robot equipped with an RTI FS-B Wrist force-torque sensor (Figure 3-2), which 
measures ail six forces and torques, The PUMA dik} has encoders at each joint to 
measure joint angles, but no tachometers. Thus, to obtain the joint velocities and ac¬ 
celerations,. the joint angleaare differentiated and double-differentiated, respectively, 
by a digital differentiating filter (Figure 3-3), The cutoff frequency of 33 Hz for the 
filter was determined empirically to produce the best results. Both the encoder data 
and the wrist sensor data were initially sampled at 1000 Hz. It was later determined 
that a sampling rate of 200 II* was sufficient,, and the data were resampled at the 
Lower rate to reduce processing time. A least squares identification algorithm was 
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Figure 3,2: Puma with a test Load, 


implemented as an uff-Sino computation, but an on-line implementation would have 
been Straightforward. 

Static Estimation Using Tho PUMA 

To test the calibration of the force sensor and the kinematics of the PUMA arm a 
static identification was performed. The forcea and torques are now due only to the 
gravity acting on the load, and equations (3.4} and (3.S) simplify to 

f = -mg (3.19) 

n = -me x g (3.20) 

Aa seen in (3.19) and (3,20), -only the mass and the center of mass cart he identified 
while the manipulator is stationary. 

To avoid needing to determine the gripper geometric parameters, the center 
of mass estimates are evaluated by the estimates of the changes in the center of 
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Figure 3.3; Meetsured angle calculated angular vielodt^ 8 t and calculated angular 
acceleration § for joint 4. 
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P arametera 

. 4 cftiai 

Static 

Dynamic 


Vafucs 

Estimates 

E’attmntej 

Mass ft 17 ) 

1 , 10 ft 

1 , 10 a 

1,067 

Change in c^fm) 

0.037 

0.037 

0.039 

Mass (ftp) 

1,106 

1,107 

1,054 

Change in ^(m) 

-0.043 

-0,043 

-0.042 

Mass (tg) 

1.106 

1.100 

1.073 

Change in -r^m) 

- 0.021 

- 0.020 

- 0.021 

Mass (jttrl 

1.106 

1.099 

1.074 

Cliange In c f (m) 

0.018 

0.018 

0.020 


Table 3.1: Mass and the center of mass estimates 

nr ms as the load is moved along the y-axis from the reference portion by known 
amounts. The results of estimation are shown In the second column of Table 3.1 
for an aluminum block {2 X 2 X Cin.) with a mass, of LlOftlfp- Only the changes In 
are shown in Table 3.1; the estimates of c. and e r remained within lmm of the 
reference values (c a = lrirtm ami c M = 47mm). Each, set of estimates were computed 
from 0 sets of data, Le. data taken at 6 different positions and orientations of the 
manipulator, where each data point is averaged over 1000 samples to minimi Be the 
effects, af noise. Tin: results show that in the static case the mass of the load can he 
estimated to within LOp of the actual mass. The center of mass can be estimated to 
within Imm of the actual values for this load. 

Static ioad estimation only tests the force sensor calibration and the position 
measurement capabilities of the robot the sensor is mounted on. In order to assess 
the Effects of the dynamic capabilities of the robot on load estimation and to be able 
to estimate the moments of inertia of the load we must assess parameter estimation 
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during general movement. 


Dynamic Estimation Using The PUMA 

In the dynamic case, the joint position Encoder and the wrist sensor data are sampled 
while the manipulator is in motion. A fifth order polynomial trajectory (a minimum- 
jerk time function) in joint space was used to minimise the mechanical vibrations at 
the beginning and the end of the movement, and to improve the Elgnal to noise ratio 
(SNR) in the acceleration data (Figure 3.3). For more popular bang-coast-bang type 
trajectories, the joint accelerations are zero except at the beginning and tbe end of 
the movements, resulting in a poor signal! to noise ratio in the acceleration data for 
most of the movement. 

We found that the PUMA robot tacked the acceleration capacity necessary to 
estimate the moments of inertia of the load, It also lacked true velocity sensors 
at the joints, which made estimation of the acceleration of the load difficult. The 
dynamic estimates of mass and center of mass for the previous load are shown in 
the last column of Table 3.1. The data used in these estimates were sampled while 
the manipulator was moving from "Q f O,D, -90, 0, 0] to [90,-60,90,90, £X),90[ degrees 
on a straight line in joint space in 2 seconds. Joint 4 of the PUMA has a higher 
maximum acceleration than the other joints, and thus, a longer path was given for it. 
This movement was the fastest the PUMA can execute using the fifth order trajec¬ 
tory without reaching the maximum acceleration for any of its joints, The estimates 
used alL 400 data paintE sampled during the 2 second movement. The results show 1 
slight deterioration in these estimates when compared to the static estimates; but 
they are still within 40^ and 2mm of the actual mass and displacement, respectively. 
However, the signal to noise ratios in the acceleration and the forec-torque data were 
too low for accurate estimates of the moments of inertia for this 2aad [0.(M23#5R r j'fn a 
in the largest principal: moment}. In this cage, the torque due to gravity is approxi¬ 
mately 4Q times greater than the torque due to the maximum angular acceleration 
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Parameters 

■ m s ) 

Actual 

V critics 

PUMA 1 

Estimates 

PUMA 1 

Estimates 

DBA 1 

Estimates 

hi 

0,0244 

D.OltfJ 

0.0246 

0,0230 

hi 

0 

-0,0048 

0,0006 

0,0006 

Aj 

0 

0,0019 

D.CHMS 

0.0005 


0,0007 

0.0021 

0,0036 

-0.0002 

J r 2 -'| 

0 

-0.0016 

-0.0004 

-0.0002 

hi 

0.(1242 

0 .0176 

0.0199 

0,0241 


Table 3,2r Actual and estimated momenta of inertia, either For all joints moving 1- or 
special test motions' 1 . 

of the load- Thus, even slight noise in. the data would resuit in poor estimates of L 
Therefore, experiments with a larger rotational load were performed for the es¬ 
timates of the moments of inertia. The new experimental load is shown in Figure 
3.2- This load has large masses at the two ends of the aluminum l>ar s resulting in 
large momenta of inertia In two directions O.024 fcrj ■ m s ] and a small moment In 
the other. A typical set of estimates of the momenta of inertia at the center of mass 
frame For the load with the gripper subtracted otit are shown in Table S -,2 for the 
above a 11-j Dints-mov 1 n g trajectory. They contain some errors but are fairly close to 
the actual values. 

Special Test Movements Using Th* PUMA 

In Order to improve the estimates by maximizing the rotational accelerations in the 
trajectories, a series of special test movements were generated. The data was sampled 
while the robot was following three separate 2 -second rotational trajectories around 
the principal axes of the load. Such trajectories used joints 4 and 6 only, and resulted 
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in higher acceleration data than the previous. trajectory, thus improving the signal to 
noiae ratio in both the acceleration and the force-torque data. Typical estimates for 
these special movements ehow improvements over the estimates with the previous 
trajectory (Table 3.2), Although the estimate of /jj is slightly worse than before, ul3 
the other terms hava improved; the cross terms, especially, are much smaller than 
before. However, these estimates of I are not as accurate as the estimates of the 
mass and the center of mase shown in Table 3.1. 

Figure 3.4 shows the comparison of the measured forces and torques, and the 
computed forces and torques from the estimated parameters and the measured joint- 
data using the simulator for the original trajectory, The two sets of figures match 
very r well even in the mechanical vibrations, verifying qualitatively the accuracy 
of the estimates. This suggcaLs that for control purposes even poor estimation of 
moment of inertia parameters will allow good estimates of the total force and torque 
necessary to achieve a trajectory. This makes good sense in that the load forces with 
the PUMA are dominated by gravitational components, and angular accelerations 
experienced by the load are small relative to those components. 

The effect of the error* causing poor estimates of moment of inertia parameters 
could be alleviated by increasing the angular acceleration experienced by the load. 
Since wa had reached the sustained acceleration capacity of even an unloaded PUMA 
robot, we decided to explore this issue using the MIT Serial Link Direct Drive 
Arm. This robot can achieve higher sustained accelerations than the PUMA and 
in addition is also equipped with tachometers at the joints, making estimation of 
acceleration much easier. 

3.4.2 The MIT Serial Link Direct Drive Arm 

The inertial parameter estimation algorithm was next implemented du the MIT 
Serial Link Direct Drive Arm (DDA), equipped with a Barry Wright Company 
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Figure 3 . 4 ; Measured force-torque data and computed force-torque d&lu from the 
estimates using the PUMA. 
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Astek FS6-12QA-2QO 6-axSa force-torque sensor which again measures all three forces 
and three torques about a point. The DDA is described in An* Atkeson, Hollerbacli 
flftBS},, and ie capable of higher tip velocities and accelerations than the FUMA. The 
DDA has tachometers Oil each of its three joints so that numerical differentiation of 
position.! is unnecessary, but we still! had to digitally differentiate the velocities to 
find the accelerations using a cutoff frequency of 30Hz, The positions and velocities 
of the robot weie initially sampled at 1kHz but were later down-sampled to match 
the sampling frequency of the force-torque sensor of 240 He, The identification 
procedure wa? again implemented off-line. 

Dynamic Estimation Using The Direct Drive Arm 

The data used for estimating the inertial parameter? of the load were sampled while 
the manipulator was moving from (280*269.1, —30) to (80,19.1*220} in one second. 
Again a Sfih order polynomial straight line trajectory in joint space was used. The 
resulting estimates for the moment of Inertia parameters are shown in the last column 
of Table 3.2, The estimates for the mass and the location of the center of mass were 
as good as the PUMA results and are not shown. We see that the moment of inertia 
parameters estimated are on the whole better than the PUMA results. 

Parameters estimated for a set of special tost movements using the Direct Drive 
Arm were not substantially different. Our special test movements for the DDA were 
not substantially faster than the movement of all joints, and thus probably contained 
the same amount of information. 

Finally, Figure 3.S shows the comparison of typical measured forces and torques 
with computed forces and torques from the estimated parameters and the measured 
joint data using the simulator for the original trajectory. Once again, we have a 
very good match between the measured and predicted forces and torques. Thus we 
see that the combination of higher angular accelerat ions and good velocity sensing 
results in better parameter estimates, as hoped. 
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F LRU re 3,5: Measured force-torque data and computed font e-torque data fram the 
estimates using the Direct Drive Arm. 
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3.5 Discussion 


3,5.1 Did The Algorithm Work? 

This- chapter describee an attempt to characterise the usefulness of wriat force-torque 
sensing for estimating the inertial parameters of rigid body load* for control and 
recognition/verification/grasping, Our conclusion is that prediction of forces for 
control can be good and seems to work well in our implementations. Identifying 
parameters well enough for recognition of the object may require large accelerations 
or spec Lai teat movements in order to accurately identify the moment of inertia 
parameters. 

It ie important to realise that there are two distinct uses of an identified model. 
For control what matters ia matching the input-output behavior of the model (in 
this case the relationship of load trajectory to load forces and torques) to r*ality 1 
while for recognition./verification what matters is matching estimated parameters to 
a set of parameters postulated for reality. We find that both of our Implementations 
of load inertial parameter estimation successfully match the input output behavior 
of the load (see Figures 3.4 and 3.3), although we have not yet used this information 
In a control scheme, HowevEr, we find that the limited acceleration capacity of the 
PUMA robot and its limited sensing do not permit ua to estimate the moments of 
inertia of the load accurately without the use of special test motions. In all cases 
Lhc mass and the Location of the center of mass could be accurately estimated from 
both a Eeries of static measurements, and measurements taken during movements. 

This work is preliminary in that an adequate statistical character Ration of the 
errors of the estimated parameters or the predicted forces has not been attempted. 
Nevertheless., we have gained insight into the sources of such errors. 


SO 


3*5 + 2 Sources of Error 


The ultimate source of error is the random noise inherent in the sensing process 
itself. The noise levels on the position and velocity sensing are probably negligible* 
and could be further reduced by appropriate filtering using a model based filter Such 
aa the Intended Kalman Filter (Gelb* 1974). The force and torque measurement 
process involve measuring the strains of structure members in the sensor with strain 
gages, The random noise involved in such measurements is also probably negligible, 

Biae Ernnv 

However, strain gages are notoriously prone to drift. We feel that periodic ^cali¬ 
bration of the offsets {very often] and the strain to force calibration matrix (often] 
may be necessary to reduce load parameter estimation errors further. Before using 
the force sensors we allowed the system to warm up and we recalibrated the offsets 
before each data collection session and cheeked for a change in the Calibrated offsets 
afterward. 

Uu modelled Dynamics 

A further source of noise is. unmodelled structural dynamics. Neither the robot 
links nor the load itself are perfectly rigid bodies. A greater source of concern is 
the compliance of the force sensor itself. In order to generate structural Stfibis 
large enough to be reliably measured with strain gages, a good deal of compliance 
is- introduced into the force sensor. The load rigidly attached to the force sensor 
becomes a relatively undamped Spring min system. The response of the Astek force 
sensor to a tap on an attached toad is shown in the “'undamped impulse response" 
record of Figure 3.6. 

The effect oF robot movement on this spring mass system is shown in the “un¬ 
damped movement response 1 * record. 


SI 


TORQUEY 



Figure 3,6r Vibrntiana of lo&dt on force eeneor. 
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There* are several approaches to deal with this problem of unmoddled dynamics. 
One approach IS to attempt to identify the additional dynamics. This greatly in' 
creases the complexity of the identification process, and greatly increases the amount 
of data that needs to he collected to get reliable estimates of any parameter. We 
feel such an approach should only he taken as a last resort. 

Another approach is to try to avoid exciting the unmodelled dynamics by choos¬ 
ing robot, trajectories that are as smooth as possible. This is one reason why we 
chose 5th order polynomial trajectories* so that we could maintain continuity of 
velocities and accelerations. Using higher order polynomials would result in even 
greater smoothness. However, we found that with the PUMA a smooth commanded 
trajectory did not result in a smooth actual trajectory* In that the control methods 
used and the actual hardware of the robot still introduced, substantial vibration. 
One way to tell if the PUMA is turned on is to Louch it and feel if it is vibrating. 
Vibrations were less of a problem with the Direct Drive Arm, although still present. 

The most successful approach is to mechanically damp out the vibrations by 
introducing some form of energy dissipation into the structure. We added hard 
rubber washers between the force sensor atid the Load* and the '“damped impulse 
response” of Figure 3.0 illustrates the response of the force sensor to a tap on the load. 
We see that the oacillalioos decay much faster. TSie “damped movement response* 
indicates that this mechanical damping greatly reduces the effect of movement on 
the resonant modes oF the force sensor plus load. Tiie conclusion we draw is that 
appropriate damping should be built into force sensors, just as accelerometers are 
filled, with oil to provide a critically damped response for a specified measurement 
bandwidth. Failing that, energy dissipation must be introduced either into the 
structural components of the robot or into the grippEr either structurally or as a 
vise dus skin. Appropriate mechanical damping may also be useful when using a 
force sensor in closed loop force control. 
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Optimal Filtering 


The need to numerically differentiate the velocity to find the acceleration, or worse 
yet, double differentiate positions, greatly amplifies whatever noise is present* One 
can avoid the need to explicitly calculate accElerations by symbolically integrating 
equations [3,4] and (3,1), One approach to integrating the equations is presented in 
the appendix, and we can express the resulting estimation equations in matrix form 


as: 
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A dr 
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However, we feel that an effort to characterize the various noise sources and 
an attempt, to “optimally” lilter and differentia Ley ini eg rale the data wilt result in 
better estimates. If there Is substantial low frequency noise or bias m the data the 
integration will amplify that noise relative to the signal frequencies of the data. 
What is needed is a problem formulation that gives us guidance as to how to design 
data processing filters for estimation and how to handle the need to differentiate 
or integrate some of the data to supply missing measurements. We are presently 
invest! gating euch an approach- 

3*5+3 Kinematic Errors 

Fart of tb« error may be due to inaccuracies in the current kinematic parameters of 
the manipulator. Experiments have shown that the actual orientation of the robot 
can be up to 4° off from the orientation computed from the encoder data. 

3.5.4 Why Estimating ZVfomerit of Inertia Parameters is Hard 

One or the factors that makes it difficult to accurately identify momenta of inertia is 
the typically large contribution of gravitational torque* which depends only on the 
mass and the relative locaLiou of the center of mass to Che fore® sensing coordinate 
origin, A point mass rotated at a radius of 5cm from a horizontal axis umsL complete 
a fufl rotation in 425 milliseconds far the torque due to angular acceleration to 

be equal to the gravitational torque* if a 5 th order polynomial trajectory ee used. A 
way to avoid gravitational torques as to rotate the load About a vertical axis, or to 
have the point of force-torque sensing close to the center of mass,. 

A simple example will illustrate the difficulty of recovering principle moments 
of inertia* given the moment of inertia tensor about the force sensing origin. The 
principle moment of inertia of a unifonn sphere ls only 2/7 of the total moment of 
Inertia when it is rotated about an axis tangent to its surface, so that the effects of 
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any errors in estimating the mass, the location of the center of mass, and the grip 
moments of inertia are amplified when the principle moment of inertia is calculated. 
This problem can be reduced by having the point of force sensing as close to the 
center of mass as possible. 

It still may be difficult to fhid the orientation of the principle moments of inertia 
even when the moment of inertia tensor about the center of mass has been estimated 
fairly accurately. This occurs when two or more principle moments of inertia are 
approximately equal. Finding the orientation of the principle axis is equivalent to 
diagonalizing a symmetric matrix, which becomes ill-conditioned when same of the 
eigenvalues are almost equal. A two dimensional example illustrates the problem; 
Consider the diagonalized matrix 


cgs(0) - sEn(0) 


1 ■ 

Ai 0 


eas(0] sin($) 

sEn(J?) cos($) 


i 

O 

u 

■ __ 


sin($J cosftf) 


[3.24) 


with eigenvalues {A In A a } and whose first principle axis Is oriented at an angle $ with 
respect to the x axis. By substituting Ai - \ 3 = c into the matrix (3.24)„ 


A 2 + £ cm j [(] e cosffi 1 ) ain(0} 
ecos(i?} sin(0) Aj + euin 1 ^} 


(3.25) 


we see that when the two eigenvalues are almost equal, the terms of the matrix de¬ 
pendent. on the angle, 0, become very Email, All terms that contain angle information 
are multiplied by tho difference of the principle moments of inertia, e. With a fixed 
amount of noise in each of the entries of the identified moment of inertia matrix, the 
orientation of the principle axis, 8, will become more and more difficult to recover as 
the principle moments of inertia approach equality and therefore e approaches zero. 
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3.6 Open Questions 


3-fi-l Data Processing Issmes 

Many of the issues discussed below will be addressed in conduction with the contin¬ 
uation of optima] filtering research described in Chapter 5. 

Statistical Characterisation of Errors 

What ia the variance of our force and torque predictions? Does this level of error 
correspond to the variability of force and torque measurements? What ie the variance 
of the estimated parameters N for a given amount of data? What measurement noise 
does this correspond kj? How close are the estimated measurement noise to the 
specifications for the force sensor? Linear least squares estimation is based on a 
particular statistical model which includes formulas for estimated parameter variance 
and predictor variances. However, it is not clear that the assumed statistical model 
is valid in this application. 

Most Efficient Movement 

Find the most efficient movement (or just a good movement) to improve the load 
parameter estimates given what is already' known about the load and which estimates 
are to be improved. This is part of a general problem in quantifying the “richness 1 * of 
the input. The statistical model assumed for linear least squares estimation provides 
a good starting point tor answering this question. 

Quantify desirable sensor properties 

What is the stiiTuepa of the wrist sensor? The Aztec wrist force-torque sensor has 
an interna] samping rate of 45011^.., a l20Tfs. analog low-pass filter, and a 120 H? 


digital low-pass filter also, for an output sampling rate of 240Hz, What are more 
■desirable sensor filtering properties? 

Kolc of Velocity Sensing 

We claim that tachometers helped with the Asada Arm, We could repeat the load 
identification without uelu g the tachometer measurements, and see how much dif¬ 
ference the tachometers made. 

Wrist sensing vs, joint sensing 

We could compare the load estimation with a wrist sensor with load estimation using 
joint sensors, and. .see if them is a substantia] difference in accuracy, 

Separate Control a oil Recognition Modules 

Since the goals of identification for control [predict input/output behavior] and 
recognition (get good object, parameter estimates] are different there should probably 
be different modules with different filtering and data handling properties. 

Effect of kinematic errors 

How robust is the load parameter estimation procedure to kinematic errors caused 
by joint measurement error and unmeasured [ink deflection? 

3*6*2 A Complete Load Handling; System 

Use rigid body load inertial parameter estimation as a single module in a more 
complete load identification system. Handle other model structures: What arc useful 
structures? What are useful parameter estimation procedures? Provide a test or 
measure for verifying that a Load is actually a rijfid body. What does this mean? 
More a rigid body than anything else? 


Note that the prediction errors (residuals) of the estimated model provide a 
natural measure for whether the body is rigid or not. 

J.6.3 Real-Time Implementation 

Implement load parameter estimation as on-line estimation procedure. It is appro¬ 
priate to address here the use of apriori load estimates. 


3-7 Appendix: Integrating The Forces And Torques 
In The Force Sensor Frame 


In this appendix wc will show how to integrate the load identification equations (3.41 
and (if.8). Throughout this derivation we will use the superscript notation a and p 
to indicate what coordinate frame, a vector m expressed in, if there is any confusion. 
If there is. no such superscript s the vector is expressed in the load frame- 
The term p is integrated using equation 7,22 of (Symon, 1971): 

R^("p) = ^(R • "W + Itr- X ”p) (3.26) 


were It is the rotation matrix representing the rotation from an inertial coordinate 
system to the force sensing coordinate system that continuously moves with the load. 
To get an integral form of this equation we simply integrate it: 


r *+T l+T ft+T 

/ R ■ D p dr = (R *p) +■ / Rfw x °p) dr 

•ft i Jt 


and performing the indicated rotations tejLves us with 
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( 3 . 28 ) 


(3.29) 
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since w x w == 0- We also remember that 



( 5 , 30 ) 


Each of the matrices ' f 4iFxj and [*w] can be integrated element by element to 


show that 



[ 3 , 31 ) 


( 3 , 35 ) 


The matrix j(g — p)x can he integrated element by element in the same way. Each 
matrix element of the terms [ f ux][ f ux] and [fWx][* p w] are numerically integrated 
by adding values at each time step, The result of this integration is given in Equa¬ 
tion (3,21), 
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Chapter 4 


Manipulator Link Inertial 
Parameter Estimation 


4,1 Abstract 

A method of estimating the masa, the location of center of mast, and the moments 
of inertia of each, rigid body link of a robot during general manipulator movement 
is presented. The algorithm is derived from the iNewton-E trier equations, and uses 
measurements of the joint torques as well as the measurement and calculation of the 
kinematics of the manipulator while it Is moving, TIls identification equations are 
linear in the desired unknown parameters, and a modified least squares algorithm, 
is used to obtain estimates of these parameters, Some of the parameters, however, 
are not identifiable due to the restricted motion of proximal links and the lack of 
full force/torque sensing. The algorithm was implemented on the MTT Serial Link 
Direct Drive Arm, A good match was obtained between joint torques predicted from 
the Estimated parameters and the joint torques computed from motor currents. 

1 Tills cliagitcT ii n, revised version of [Alt, AtkeSdli, and Hs&JJtrbadl, ]URfia) 
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4,2 Introduction 


This chapter presents a method of estimating all of the inertial parameters, the 
mass, the center of mass, and the moments of inertia of each rigid body link of a 
robot manipulator using joint torque sensing. Determining these parameters from 
measurements or computer models is generally difficult and involves some approxi¬ 
mations to handle the complex shapes of the arm components. Typically, even the 
manufacturers of manipulators do not know accurate values of these parameters. 

The degree of uncertainty in inertia] parameters is an important factor in judging 
the robustness of model-based, control strategies. A common objection to the com¬ 
puted torque methods, which involve full dynamics computation (e.g., Luh, WaJkcr, 
and Paul, XBSOb), is their sensitivity to modelling errors, and a variety of alternative 
robust controllers have been suggested (Samson, IDS 3; Sloline, 1985; Spong, Thorp, 
and Kleinwaks; 1984, flilhert and Ha, lffcS4). Typically these robust controllers ex¬ 
press modelling errors as a differentia] inertia matrix and coriolia and gravity vectors, 
but in to doing, no rational basis Is provided for the source of errors or the bound? on 
errors. The error matrices and vector? combine kinematic and dynamic parameter 
errors, but kinematic calibration Es sufficiently developed ao that very little error can 
be expected in the kinematic parameters (Whitney, Lorinski, and Rourke, 1084), 

One aim of this work is to place similar bound? on inertial parameter errors by 
explicitly identifying the inertia] parameters of each link that go into the making of 
the Inertia matrix and Coriolis and gravity vectors. Our work in load identification 
(Atkeson, An, and Hollerbach, 1985a) suggests, for example, that mase can be accu¬ 
rately identified to within 1%, Therefore, an assumption of 50% error in link mass 
in verifying a robust control formulation (Spong, Thorp, and Kleluwaks, 1954) 5a an 
unreasonable basis for arguments Slotine (1985) suggests that errors of only a few 
percent in inertial parameters make his robust controller superior to the computed 
torque method, but It may well be that these parameters can he identified more 
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accurately than his assumptions. 

As an alternative approach wa propose estimating the inertial parameters on the 
basis of direct dynamic measurements. The s am e alEoritlnns used to identify load 
inertial parameters (Atkeson, An s and FTollerbajcb, 1985ft) can be modified to find 
link inertial parameters of a robot arm made up of rigid parts. The Newton-Euler 
dynamic equations are used to express the measured, forces and torques at each 
joint In terms of the product of the measured movements of the rigid body links 
ami the unknown Link inertial parameters. These equations arc linear in the inertial 
parameters, However, unlike load estimation, the only sensing is one component 
of joint torque, inferred from motor current. Coupled with restricted movement 
near the bane,, it ig^ therefore,, not possible to find all the inertial parameters of the 
proximal links. As will be seen, these missing parameters have no eEFert on the 
control of the arm. 

In this chapter, manipulators with only revolute joints are discussed since han¬ 
dling prismatic joints requires only trivial modifications to the algorithm. The pro¬ 
posed algorithm was verified by implementation on the MIT Serial Link. Direct Drive 
Arm. 

4.2.1 Previous Work 

Mayeda, Qsuka, and Kangawa (13hl) required three sets of special test motions to 
estimate the coefficients of a closed-form Lagranglan dynamics formulation. The 10 
inertial parameters of each link are lumped into these numerous coefficients, which 
arc redundant and susceptible to numerical problems in estimation. On the Other 
hand, every coefficient is identifiable since these coefficients represent the actual 
degrees of freedom of the robot. By sensing torque from only one joint at a time, 
their algorithm is more susceptible to noise from transmission of dynamic dfecls of 
distant links to tlic proximal measuring joints. For efficient dynamics computation, 


(13 


the recursive dynamics algorithms require the link parameter* explicitly. Though 
recoverable from the Lagranglan coefficients, it is better to estimate the inertial 
parameters directly. Though this algorithm was implemented on a PUMA robot , 
it is difficult to interpret the results because of dominance of the dynamics by the 
rotor inertia and friction. 

Mukerjee and Bayard [19SS} directly applied their toad identification method tn 
link identification, by requiring ful! force-torque sensing at each joint. Instrumenting 
each robot link with full force-torque sensing seems Impractical, and is actually 
unnecessary given joint torque sensing about the rotation axis. Partially as a result, 
he docs not. address the issue of unidentifiability of some inertial parameters, Also, 
he did not verify his algorithm by simulation or by implementation. 

Olsen and Bekey (1955) presented a link identification, procedure using joint 
torque sensing and special test motions with single joints. The unidentifiability 
of certain inertial parameters waa not resolved, and the least squares estimation 
procedure written as a generalized inverse would fail because of linear dependence of 
some of the inertial parameters. Again, their procedure was not tested by simulation 
or by actual implementation on a robot arm, 

Neuman and Khosla (i&SS) developed a hybrid estimation procedure combining a 
Newton-Euler and a Lagrange-Euler formulation of dynamics. Simulation results for 
a three degreo-of-freedom cylindrical robot were presented, and the unidentifiability 
of certain inertial components was addressed, Tor some reason they state link mass 
must be known for a linear estimation procedure, but such a restriction does not 
exist with our method. Though planning to work with the CMTJ DDArm U, they 
have not yet presented experimental re&ultE, 
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link i 


joint 1+1 


y 

n 

Figure 4.1: Coordinate origins and location vectors for link identification, 

4,3 Estimation Procedure 

4.3.1 Formulation of Newton-Euler Equations 

Til our work in load identification (Afkeson, An, and HoLter-bach, 1985a), the Htrwton- 
Emer equations for a rigid body load weK formulated to be linear in the unknown 
inert iai parameters, Then tin® simple linear least squares met hod was used to es¬ 
timate those parameters, fiy treating each link of a manipulator as a toad, this 
for mutation can be extended to the link estimation problem. The differences in the 
equations are that only one component of force or torque is sensed and that the 
forces and torques from distal links are summed and transmitted to the proximal 
joints. 

Consider a manipulator with n joints [Figure 4-1], Each link t has its own local 
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coordinate system Pi fixed in the link -with its origin at joint i. The joint force and 
torque due to the movement of Its own link can be expressed by simply treating the 
link as a toed and applying the previously developed equations for load identification 
[Atkeson, An, and nollerbach, IUB5a): 


"td 

miC ri 


™f*Vi 


* 


Pi-K [di^x] + fhPjXjjWpXj 0 





0 [(k ” Pi) x] |*w] + kj£ r x|J*W;] 






or more compactly. 


W[£ = 


( 4 . 1 ) 


where w^- ia the wjench (vector of forces and torques) at joint i due to movement of 
link j alone. A* is the kinematic matrix that describes the motion of Link t and ^ is 
the vector of unknown [ink inertial parameters. All of the quantities are expressed 
in the local joint i coordinate system, The formulation of the above Newton-Euler 
equations were already presented In the load identification paper (Atkeson, An, and 
Hollerbachp l985a) t 

The total wrench w, at joint i is the sum of the wrenches for all links j distal 
to joint i: 

Wi = J2wii ( 4 . 2 ) 

/=« 

Each wrench w,-,i at joint i i& determined by transmitting the distal wrench across 
intermediate joints. This la a function of the geometry of tlie linkage only. The forces 
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and torques at neighboring joints are related by 


t'.hi 


Ri 

0 



Tl^+i ^ 


|*tx| -Ri 

R< 




or more compactly 


(4,3) 


W i,i +1 — Tj W dl+| p t+l 

where 


(4,4) 


31,= the rotation matrix rotating the link i 4- 1 coord in ate ayatem to the li: jk i 
coordinate system, 

6,"— a Vector from the origin of the link i coordinate system to the lint t" Hi-1 coor¬ 
dinate system, and 

Tt= a wrench transmission matrix. 


To obtain the forces and torques at the t fik joint due to the movements of the j* k 
Eink, these matrices can be cascaded: 


w O~T,T ++ |... TyWjiji 

=U*#i 


(4.S) 


where U^- = T,T,+ |’ '’TyA, and TJ;, = A,. A simple matrix expression for a serial 
kinematic chain {in this case a aix joint arm) can be derived from (4,2) and (4,5]: 
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This equation is linear in the untnowu parameter, but the left side is composed 
of a full force-torque vector at each joint. Since only the torque about the joint 
axis can usually he meASUred, each joint wrench must be projected onto the joint 
rotation axla (typically [0*0,1] in internal coordinates), reducing (4.fi) to 

{4,7} 

where tv — |0 r 0,0 r 0,ft,1 ] - W; is the joint torqua of the t* h [ink t $ — j •. i 

and K,j = [ 0*05 0,0* 0*1] ■ when the corresponding entry in {4,6} is nonzero. For 
an ti-link manipulator, r is a a x 1 vector* $ is a lOn x l vector* and It is a rt x lQn 
matrix. 


4.3.2 Estimating the Link Parameters 


Equation (1,7) represents the dynamics of the manipulator for one sample point. 
For extra data* (4.7) ia augmented ASi 


K .= 

‘ K(l) J 

T — 

' Ml) ‘ 






jV — number of data points 


Unfortunately, one cannot appEy simple least squares estimator 


= (K T K) -1 K T r (4.8) 

because K r K is not invertible due to loee of rank from restricted degrees dF freedom 
at Lhe proximal links and the lack of full force-torque sensing. 

There are several ways to resolve this problem. One way to resolve this problem is 
to use the pseudo inverse to get a solution to T = K$. But since K is a potentially 
lar^e hjV x lQn matrix, the pseudo-inverse is computationally inefficient. Another 
simple method similar to the pseudo inverse is to use ridge regression {Marquardt 






and Snee, 1975). Ridge regression makes K 7 K invertible by adding a small number 
to the diagonal elements: 

* = (K r K + dI iqw ) l K ? r (4.9) 

The estimates are nearly optimal if d « A^JK^K), where is the smallest 
non-zero eigenvalue of K. T K, Other methods of solution,, fundamentally different 
from the above two., Eire presented in the discussion section. 


4,4 Experimental Results 

Linli estimation was implemented on the MIT Serial Link Direct Drive Arm (Figure 
4.2) > a three link serial manipulator with no transmission mechanism between the 
motors and the links. The ideal rigid body dynamics is a good model for this arm, 
uncomplicated by joint friction or backlash typical of other manipulators. Hence the 
fidelity of this manipulator’s dynamic model suits estimation welt. The coordinate 
system for this arm is defined in Figure 4.3. A set of inertial parameters is available 
for the arm (TVb|e 4.1 ) p determined by modeling with a CAD/CAM database (Lee,, 
1983). These values can serve as a point of comparison for our method, but they 
may not be accurate because of modeling errors. 

The motors arc rated at GEO Nm peak torque for joint 1 and 230 Nm for joints 
2 and 3 (Asuda arid YouceTToumi, 19S4). Joint I is presently capable of an an¬ 
gular acceleration of HoQ dcgfse c 1 , joints 2 and 3 in excess of fiflfWJ dep/sec s - In 
comparisionj joint 1 of the PUMA fiflO has a peak acceleration of ISO deg/sec* and 
joint 4 at the wrist 260 dcgfscc 1 . Joint position is measured by a resolve:- and joinL 
velocity by a tachometer. The tachometer output is of high quality and leads to 
good acceleration estimates after differentiation. The accuracy of the acceleration 
estimates plus high angular accelerations greatly improves inertia estimation. 

The joint torques are computed by measuring the currents of the 3 phase windings 



Figure 4.2: The MIT Serial Link Direct Drive Arm 


TO 




Figure 4,3; The link coordinate system. 
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Parameters 

Link 1 

Link 2 

Link 3 


67.13 

53.01 

19.67 

muffin ■ m) 

ao 

0.0 

0.3106 

rrtCy 

2.432 

3.4081 

0.0 

me. 

35.8257 

16.6305 

0.3268 

/«[*?■«■»*) 

23.1S68 

7.9088 

0.1825 

/* i> 

0.0 

0.0 

0,0 


-0.3145 

0.0 

-0.0166 


20.4472 

7.6766 

0.4560 


-1.2948 

■1.5036 

0.0 

A_ 

| 0.7418 

0,6807 

0.3900 


Tabic 4.1; CAD-modeled inertial parameters. 

uf each, mol Dr (Asada, Ycmcef-Toumi, and Lim, 1984). For the three phase brushless 
permanent magnet motors of the direct drive arm, the output torque is related to 
the currents in the windings by r 

t = K^ih =tn 8 + h 4- 120) + 7 C sin(0 + 240)) [4.10] 

The torque constant Kj for each motor is calibrated statically by measuring the 
force produced by the motor torque at the end of a known lever arm. The force 
is measured using a Barry Wright. Company Astek FS6-10A-200 6-axis force/torque 
sensor. Asada, Yaucef-Taumi, and him [1984] have found that for A motor simitar 
to the motors of our manipulator;, the torque versus current relationship was non- 
Linear, especially for small magnitudes of torques, and also varied as a function of 
the rotor position. However, for the results presented in this paper, the nonlinear 
effects were ignored since substantial portions of the movements in the experiments 
required large magnitudes- of torques. Since the least squares algorithm minimizes 
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the square of the error, torque errors for torques of small magnitudes do not affect, 
the estimates very much. 

For the estimation rremEts presented, WO data points were sampled while the 
manipulator was executing 3 sets of fifth order polynomial trajectories tn joint space. 
TEie specifications of the trajectories were: 

1* (330, 289. 1, 230) to (ao, 39,1, -10) degrees In 1,3a, 

2, (330, 269.1, -30) to (SO, 19,1, 220) degrees In 1,3s,. 

3, (SO, 209.1, -30] to (330, 19,1, 220) degrees in 1,3s, 

Since K r K in (4.9) is singular, estimates for the 30 unknowns, are computed by 
adding a small number (rf — 10,0 << A ffl ., rt (K r K) = 3395.0) to the diagonal elements 
of K T K. 

Typical results, obtained using the ridge regression method, are shown in Table 
4,2, Parameters that cannot he identified because of constrained motion near the 
base are denoted by 0,0’. The first nine parameters of the first link are not identi¬ 
fiable because this link has only one degree of freedom about its z axis. These nine 
parameters do not matter at all For the movement of the manipulator and thus can 
he arbitrarily set to 0,9. 

Other parameters marked by (f) can only be identified in linear combinations, 
indicated explicitly in Table 4.3, The ridge regression automatically resolves the 
linear combinations in a least squares sense, It can be seen that the estimated sums 
roughly match the corresponding sums Inferred from the CAD-modeled parameters, 
but the sizeable discrepancy indicates that one parameter set may be more accurate 
than the other. 

To verify the accuracy of the estimated and the modeled parameters* the mea¬ 
sured joint torques axe compared to the torques computed from the above two sets 
of parameters using the measured joint kinematic data. As shown in Figure 4.4, the 
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Figure 4,4; The measured,, CAD-modelled, and the estimated joint torques 
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estimated torques match the measured torques very closely. The torques computed 
from the CAD/CAM modeled parameters do not match the measured torques as 
closely. This comparison verifies qualitatively that for control purposes the esti¬ 
mated parameters are in fact more accurate than the modeled parameters. 

4,5 Discussion 

Good estimates of the link inertial parameters were obtained, as determined from 
the match of predicted torques to measured torques. The potential advantage of 
this movement-based estimation procedure for increased accuracy as, wel] as con¬ 
venience was demonstrated by the less accurately predicted torques based on the 
GAU-modcIcd Inertial parameters. 

There are three groups of inertiat parameters: fully identifiable, completely 
unidentifiable, and Identifiable In linear combinations. Membership of a parame¬ 
ter in a group depends on the manipulator’s particular geometry, Some [ink inertial 
parameters are unidentifiable because of restricted motion near the base and the 
lack of full force-torque sensing at each joint. For the first link, rotation is onty 
possiblc about its a axis. Suppose full forte-torque sensing is available at joint 1, It 
can be seen from ( j il] that hi t , J =?u and I iyj are unidentifiable because they have 
no effect on joint totque. Since the gravity Vector is parallel to the 2 axis, e Jl is- also 
unidentifiable. If It is now supposed that only torque about the z axis can be sensed, 
then all inertial parameters for link 1 become unidentifiable except 

Jn a multi-link robot a new phenomenon arises. Some parameters can only be 
identified in linear combinations, because proximal joints must provide the torque 
sensing to identify fully the parameters oF each link. Certain parameters from distal 
links are carried down to proximal links until a link appears with a rotation axis ori¬ 
ented appropriately for completing the identification, I]] between, Lhesiz parameters 
appear in linear combinations with other parameters. This partial idontiliability and 


75 


the difficulty of analysis become worse as the number of links are increased. 

The ridge regression automatically resolves the linear combinations in a least 
squares sense, which make these inertial parameters appear superficially different 
from, those derived by CAD modeling. An alternative method is generation and 
examination of the closed-form dynamics, which is a complex, procedure for more 
than two degrees of freedom. 

A second alternative is a numerical analysis via singular value decomposition of 
K in (4.8), yielding (Golub and Van Loan, 1981) 

K = UEV t 

where H = diag{o , j } and IT and V T are orthogonal matrices, For each column 
of V there Corresponds a singular value tf, which if not zero indicates that linear 
combination of parameters, vf^, is identifiable. Since K is a function only of the 
geometry of the arm and the commanded movement, it can be generated exactly 
by Simula Lion rather than by actually moving the real arm and recording d ata with 
inevitable noise. 

The above two procedures isolate several sets of parameters whose linear com¬ 
bi nations within each set are identifiable. Then we can arbitrarily set all but one 
of the parameters, of each set to zero and apply the estimation algorithm for the. 
reduced set of fully identifiable parameters. As shown in Table 4,2, for our 3 link 
manipulator, these two procedures result In grouping She 3D Inertial parameters into 
the following categories: 

1. fully identifiable: I IIai J Ufa , T ati , m 2 c^ f I rvt> 

2. completely unidentifiable: m s , msc^, m u hilC,,, mjc,,, J,„ |t 

A tVi i J vn 

3 . identifiable in linear combinations: m 3 , I 1IJt T m , m 2 c„ in i«*, I„ 3t I tia3i 

fe± t n 
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Parameters 

Link 1 

Link 2 

Link 3 

ro(Jfs) 

O.O' 

0,0' 

1.3920* 

F7tC a (/i"5 ■ m) 

0,0" 

-0.1591 

0.4676 

me,. 

0,0* 

0.67761 

0.03 IS 

me. 

0.0* 

d.o* 

-1.00871 

I a± (Kq^m 2 ) 

o.cr 

4,15G2t 

1.5276f 

h v 

0.0" 

0.3604 

-0025 b 

h. 

0,0* 

G.oua 

O.0143 

*v\t 

0,0* 

5.21291 

1.89677 


0.0* 

-0,6050t 

-0.0160 

f IZ 

9.335tWt 

-0.8194t 

0.356S 


Table 4.2 ; E^timsited Inertial parameters. 

The completely unidentifiable parameters can be arbitrary set to zero. For the 
linear combination parameters, the combinations of these parameters that can be 
identified together are shown in Table 4,3, To obtain a particular solution for these 
parameters, one can set / Brat and I IZi to s^ro. Then, the remaining 6 

parameters of this category can be treated as fully identifiable parameters along 
with the ft parameters m the first category. Those columns, of K which correspond 
to the 15 zeroed parameters are taken out, reducing K r K to a 15 X 15 full rank 
matrix. Now, simple least squares can be applied to estimate the If? identifiable 
parameters. For our implementation experiments, the results of this method agreed 
with the results of ridge regression presented in the previous section. 

Although not as simple as the ridge regression method, these methods are attrac¬ 
tive since it allows us to reformulate the dynamics in. terms of only the identifiable 
parameters. This then can increase the efficiency of the corresponding inverse dy- 














linear Combi nations 

Estimated 

CAD-Modeled 

-l- l ur3 

4.0589 

4.3565 

L — 

-0.3691 

-0.2702 

1**3 "I" 

0,7082 

0,8632 

I** t + h*i + hxi + ntj/f 

15.7029 

12.8169 

1*3 ;■* “h 

0.4 7G9 

0.4147 

rni^yi 

4.6863 

-3-0814 


Table 4,3: Parameters In linear combinations (fj = 0,45m.) 

namtes computation for controller implementations [Halkrbach and Sahar, 1983). 
We are still Investigating these Irenes with, eventual goals of studying the effective¬ 
ness oF different control algorithms Using the cstimated dynamic model and analysing 
their robustness with modelling errors. 


4.6 Open Questions 

Statistical characterization oF the errors, finding “rich* movements, quantifying de¬ 
sirable sensor properties and the importance of velocity sensing are important issues, 
just as in the rigid body load parameter estimation work. 

The effect oF deviations from rigid body dynamics caused by kinematic mod¬ 
elling errors! link flexibility, and unmodelled joint or actuator dynamics needs to he 
examined. Tor the direct drive arm we have verified experimentally the usefulness 
of the rigid body dynamics model, however. By Identifying the inertial parameters 
directly rather than estimating them from a GAD/CAM model or the properties of 
the individual links we inadvertently compensate for some model structural error. 






Chapter 5 


Optimal Filtering For Parameter 
Estimation 


5*1 Introduction 

Often in pit a met it estimation there is a substantial amount of noise contaminating 
all of the sources of data. Furthermore^ in many estimation situations them are vari¬ 
ables in the parameter estimation equations Lhat are not measured directly, but are 
known transformations of other measured data. These factors present problems for 
many parameter estimation approaches, such ns least squares.. This chapter presents 
a three step approach to these problems. Thu first step makes use of redundancy in 
the Effusing to characterise the noise sources. The second step makes use of the noise 
characterization to design optimal filters for the different data sources. Finally, an 
eigenvalue /eigenvector based estimation procedure is used to produce the parameter 


estimates. 


5.1*1 Problem Statement 


The typical Linear parameter estimation problem is of the following fomu 

«i*i + = 0 [&-!) 

or equivalently in vector form 

x r ■« = 0 (5.2) 

w?th if being measurements from various data sources and the ^ being the un¬ 
known parameters. Measurements from the data sources are each contaminated 
with statistically stationary zero mean random noise with variance 

The parameter set, is actually redundant, as It can be scaled 

without changing the problem* so typically either the length of the parameter vector 
is constrained, or one of the parameters is set to same arbitrarily chosen value. We 
will require that the length of the parameter vector is 1, ia. t 

E*i”i M) 

■ml 

Many data points consisting of sets of observations ,r n } can bE col¬ 

lected. The goal here is to find a set of parameters, (oi, oj f r ,., cr rt ^, that obey any 
constraints such as a fixed parameter vector length, and also in some optimal sense 
provide a best fit to the data. We will disc use what we mean by optimal In what 
follows. 


5*1*2 The Least Squares Approach 

One approach that is widely used is known as least squares (Silvey, 1EJT5). A crucial 
feature of this approach, is that all but one of the data sources are assumed to he 
noise free, such that the problem is restated as followsr The constraining problem 
structure is now; 

«1*1 + a*** + " ■ + frn-l*,,-! = *„ [5-4) 


an 


whef£ a n lb arbitrarily set to — l and 


£„ - X m - € 


(5.S) 


in is the true value of and e La the measurement error. The variance of the 
contamination noise, tr u is zero for a]l i < n. AH of the noise jg assumed to come 
from the variable t> which ts a zero mean random variable with variance cr H . The 
equivalent vector form for Equation {5,4} is 

x r ' a 4-« — J„ (5.6) 

If ail of the p data points collected are used to build a data matrix. A, and an 
measurement vector, y* the following matrix equation can be written: 


A - oe 4- c = y 

with the data matrix A being the p x [n — 1) matrix 



3 11 


* +- 

A = 


*i 3 

L *■ 

■ ■ 



X U 



and the measurement vector f being a vector of length p- 


y = 


i \ 


(5.7) 


(5.S) 


f *■. ) 

The criteria to be minimized by the chosen set of parameters* {aj,, . 

is the length of the error vector* e* ie. 

P 

i=i 


(5.S) 


* On— | h 


(5.10) 


81 






The estimate that minimize* this ■criteria Is given by the normal equations: 

4=(A r A)- 1 A T y (5,11) 

This approach works welL on many estimation problems, even though in the real 
world no measurement!* are perfect- For some problems, however, the contaminating 
noise on the different, data sources is substantial, and cannot be ignored. In this case, 
* different approach is necessary. 


B*l*3 Motivation For A Different Approach: A Typical Es¬ 
timation Problem 


Recently, we have addressed the problem of estimating the Inertial parameter* of & 
rigid body load being manipulated by a robot equipped with a wrist force/torque 
sensor (Atkeson, An and Hollerbach, !9$&a n I0S5b). We ware able tp formulate the 
rigid body dynamics in the following formr 


ft 

A 

A 

n B 


p-g ;us<] +'wxj|wx] £> 

0 l(8-P)*j [*w] + [w*][H 


m 

mc e 

me* 

Jii 

A3 

Ai 

hi 


( 6 - 12 ) 


hi 

Ai 

The derivation of this equation and a definition of the notation is provided in the 
Appendix, The important point is that this problem can be formulated as a linear 
parameter estimation problem. In that known par a met era enter non-tincarly, but 
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the unknown parameters, only enter linearly in the ■determination, of the forces and 
torques on the load. Equation {5.12) can he more compactly written as 

w = MflJ (5.13) 

where w is? a wrench vector, a vector of three forces and three torques, M is a 
matrix describing the movement of tlic load, and ^ is the vector of unknown inertial 
parameters. 

We noted that the unknown parameters appeared El nearly in the problem, and 
therefore Wt thought that the ieast squares approach to estimating the parameters 
could be used. In order to apply the least squares approach to this problem we had 
to measure the robot position, velocity, and acceleration, and the forces and torques 
on the wrist sensor, Furthermore, the position, velocity, and acceleration must 
he measured perfectly. Unfortunately, this was not a reasonable model of reality, 
The robot was only equipped with position sensors, and we had to numerically 
differentiate the position data to find the velocity and acceleration. To do this We 
used a discrete differentiating filter convolved with a low pass filter (Hamming, 1977), 
A typical sample of the data is shown in Figure 5-1 ► The differentiation process 
amplified whatever noise was in the position record, leaving, the derived acceleration 
record greatly contaminated with noise. The force and torque data was also clearly 
contaminated with noise (Figure 5,2}- 

Thus, this problem did not easily fit into the standard least squares framework, 
because all of the data sources have substantial contaminating noise. 

One possible approach to handling the noise is to filter the data. However, 
whatever filter that is applied must preserve the relationship between the different 
data sources, and we need to know which filter gives us the best parameter estimates. 
The standard Least squares parameter estimation approach gave us no guidance as 
to how to filter the data to achieve these goals. 

Another approach to this problem is to symbolically integrate the load param- 



Figure <5.1: Example of measured joint, angle, calculated joint angular Velocity, and 
calculated joint angular acceleration. 
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Figure 5-2: Example of measured forces and torques. 


cter estimation equation. Equation (5.12), so as to avoid differentiating velocity to 
find accelerations. One approach to integrating the equations is presented In the 
appendix to this chapter, and we can express the resulting estimation Equations in 


matrix form as: 


m 

mc t 


me,, 



(sit) 
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where the first row of [j/^Mdr] lb 

t+r j-i+r / /t+r \ 

P + jf wXpd>-|/ RdrJ^ 


7 1 *+^ ' 

rt+r 

- 

) x ] 

+ j [ux][w>c]dr 

Q 


[1SJ 


and the second row is 


{J ^-p‘p -f* T uXvdr+^J* +T Rdry^ xj 

(•OH 


(+r 


[wxj[*w] dr 


(16) 


It Is not clear, however, that this is- the right thing to da. Tf there is substantial 
low frequency noise or bias in the data the integration will amplify that noise relative 
La the signal frequencies of the data, What is needed is a problem formulation that 
gives us guidance as to how to design data proteEsing filters for estimation and how 
to handle the need to differentiate or integrate some of the data to supply missing 
measurements. 


5*1*4 Prototype Problem To Be Discussed As Example Of 
Procedure 

The problem we are going to focus on in what follows as the load inertial parameter 
estimation problem stripped down to its bare essentials. We have a ane-dimensional 
syst-Eir, and ita dynamics are given hy: 

/ = trt ■ a {5. IT) 

which is simply force = mass ■ aecflfcrflft’itsn. 

Our goal is to Estimate the mass of the object as accurately as possible, given a 
fixed amount of data. Unfortunately, we only measure the force and the velocity 
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of the. object. We wish to determine the data processing procedure to that produces 
the best estimates, in some sense. We have several choices: 

* Differentiate velocity Lo estimate acceleration, and use the equation 

/(f) - (5,18) 

to produce the estimates. 

* Integrate the forte, and use the equation 

/ 1 /(<)<** = ™{v (t/) - v(*r)) (SJ&) 

to produce the estimates. 

* Differentiate velocity to estimate acceleration, but then apply a filter of some 
sort to ‘‘dean Up 11 the data. The estimates are produced from an equation dF 
the form 

!*/(#)= m{U±)*v{t) [5.20) 

where J is some filter and + is the convolution operator. 

The goal of this chapter is to show which of these choices is the best choice to 
make, and show what factors affect the choice of data processing procedure. To do 
this, we will start by explaining how an estimation procedure that can handle noise 
in all the data works. We will first show how standard least squares sr applied, and 
we want to look carefully at what criteria it optimizes and how It handles colored 
noise. Next we shall examine an eigenvalue/eigenvector estimation procedure, how 
It applies to noise on all data ca-W, and bow it handles colored noise. We will explain 
how the Colored noise estimator can He partitioned into a white noise estimator 
preceded by appropriate data filters. Then we will ask what data processing produces 
the best estimates from tins estimation procedure, and then we wilt show how to 
generate the information about the data and the noise that allows us to design the 
appropriate data processing procedures. 
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5.2 The Estimation Procedure 


We will now discuss application of the estimation procedure to the prototype prob¬ 
lem, estimating the mass of a one-dLmenstonali load. First, we must describe notation 
for all the quantities we need to keep track of. /{t), a(t), and u(t) are measured or 
estimated forces, accelerations, and vetocities. /(i), d[f) T and D(f) are the underly¬ 
ing true values for these quantities, m is the true mass of the object, and m is the 
estimated mass. Note that 


/(f) = m-h{t) ™ m ^ v{t) 

holds for the true values of the data, but since 

(5,31) 

/(0«/W + */W 

(5,22) 

(t{i) - a[f) + n*(f) 

(5-23) 

wfO - *(*} + 

(5.24) 


whore Fi/fi), n. a (f], and n u [t) are the noise contaminating the data, the corresponding 
equalities for the measured data do not hold; 


/(*) ^ m - a [t) t n ■ —irfi) { 5 . 25 ) 

5*2*1 Application Of Standard Least Squares To Prototype 
Problem 

What Is presented in this section is well known (Silvey, 1975, for example) and is 
presented so as to make what follows more comprehensible. 

To show how to apply standard least squares to this problem, wc must assume 
wo have a perfect acceleration measurement, a(t) = £((), but a noise contaminated 
force measurement, /(f) = miff) — nj[t)< We will model the noise, as aero 


S8 


mean, gaussiau random noise, n f (t} is not necessarily ‘"white 11 noise, however, hut 
can have any power spectrum, denoted by (w)* 

We can estimate m using standard least squares in the following way. In the 
white noise case, = 1, the appropriate criteria to minimise lb 


E<’«) 


where 


c{*) = m - a(f) - /(f) 

We express the data in matrix/vector form = f -(- c 


.(1) 1 


' m ’ 


f ■(!] ) 

«(2) 


m 

+ 

e(2) 

L a(p) j 


l m j 


l f{*J J 


and the solution is given by the normal equations 

£ = (A^AJ'A 1 / 


(5.26) 


(5.27) 


(£.23) 


(5.29) 


The standard least squama approach minimizes errors along only one coordinate 
direction (Figure 5.3A). This should be contrasted with the eigenvalue/eigenvector 
approach. 


Effect Of Colored Noise 

The effect of colored noise (the noise spectrum S nf is not constant) is to lead us 
to weight the errors in our criteria for an optimal estimate. To show this we must 
analyse how our parameter estimation schemes, work in the frequency domain. 
Since 

t(f) = m <t(() - f(t) - »/(*), (5.30) 
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the power spectrum of the noise, S 9 , is simply the power spectrum of the noise 
contaminating the force measurements, V In the white noise case (the noise 
spectrum S ri is constant) we wanted to minimize the criteria 


E<’[<) = B"> “W - /(*»)’ (5.31) 

I t 

and by Parseval’s Theorem (Oppenheim and Wilsiy, 


E^t 1 ) = E l-E(«)l* (5.32) 

i w 

where iT(w) is the Fourier transform of f(£). Since the expected spectrum of the noise 
was Elat* it was reasonable to weight the error components at different frequencies 
equally. 

In the colored noise case weighting the errors equally at different frequencies no 
longer makes sense. Since the power spectrum of the noise is S„ f (cj) ami is not 
constant, we should weight the error components at different frequencies according 
to the expected amount of error at that frequency. Our optimal estimation criteria 
to minimize becomes [in the frequency domain} i 


Y' l*MP _ r-1™ • ■*(«) - f Ml' 

t4>) “V -?■» 

This is equivalent to prefiitering the data with the frequency domain filter 


(5.33) 


1 



or convolving the data with the inverse Fourier transform of this filter. 




5.2*2 Handling Noise On All Measurements 

The eigenvalue/eigenvector parameter estimation procedure presented in this section 
was originally developed by (Koopmans, 1337), and (Levin, 1964) made use of it in 
pulse transfer function estimation. Many others have extended it in various ways: 


91 




(Smith,, lftfifl, Bonlvento and Guidorzi, 1971, Groejean and Foulard, 1978, Kotta, 
197ft, Tian-Zing, et. al t 19S2, Kotta, 1932) 


Now Jet ue assume there is noise on both the force, ?i/, and the acceleration n it 
measurement; 


/[<) = /(() - n/ (t) (s.as) 

a(f] - d(t} - n(t)i (5.30) 


Both noise sources are sere mean, gauaslan, and with power spectra and 
reppectivcly. The noise sources are independent of each other, also. Initially, let us 
assume that variance of n a and the variance of nj are equal, and the noise sources 
are both white. 


Now we need ta reejcajoine what the error is that Wfi am miijiinUinj. We Bee in 
Figure S.3B that the error we now want to minimtEe ia in the direction of the vector 


*' 

-1 


and for a data point (a(f) ,/(#)) T the error is given by 

e±± Ao[()“/(*) 


( 5 , 37 ) 



(5.38) 


Thi* Ea the component of the ‘‘true’ 1 error perpendicular to the relationship /[() 
ffr ■ e(f] . 

We would JiJte the estimate fh to minimize 

T*' ;/ f i| _ ‘ fl ( e ) “ /(OJ* 

V i; i A*+3 


(5.39) 


This is a nonlinear minimization problem since the desired estimate, m appears in 
both the numerator and the denominator of the objective function, and is squared 
in the denominator. Thus, we- cannot apply standard least Equates techniques to 
this problem, or use the normal equations to find the estimate. 







Au Eigenvalue/Eigenvector Based Estimation Procedure 


We can Rolve this nonlinear estimation probtem if we think of it as a linear algebra 
problem (Koopmans, 1W), We express efi) as A vector: 


“(1) /(l) 1 

f * ) 

•CD ) 

a(2J /(2) 

I” 1 ) 


J m 

L a fp) ft.P-) j 

(m> + l)i/s - 

► 

p 

b 

l f(p) J 


and notice that the above forms the matrix/vector equation: 


[5.4Q) 


l*v-e (5,41) 

Note that v has unit length, 

Our estimation problem is now to find the unit length vector v that ruinimije& 
the length of By, This wilt minimize 


!>’(*} (5.42) 

t 

as we desire. Now that wc have reformulated the estimation problem as a linear 
algebra problem, we can recognize that the desired v is simply the light eigenvector 
of B corresponding t -0 the smallest singular value of B, or equivalently, V la the right 
eigenvector corresponding to the smallest eigenvalue of B r D [Golub and Van Loan, 
19S3), To see this, we realize that minimizing the square of the errors is equivalent 
to minimizing 

=r* 7 ’B r B* (5,43) 

t 

We merely decompose a candidate V into its projection along the orthogonal eigen¬ 
vectors of R r B, i.e. f 

* - 7e v i + TiVj + ■ ■ ■ + VV [&-44) 
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where Vt Is the eigenvector corresponding to the smallest eigenvalue A^ Since v is 
of unit length, = t. The function to minimize is now 

v T B r Bv = AM + AJ-tiJ + - ” + AJtJ (5,45) 

To minimise this sum we should choose v — Vi, as previously stated, m can be 
easily computed from ft, although it may be hard to find the error distribution of m 
given the error distribution In V, 


Effect Of Colored Noise 


The effect of colored noiEe is ones again to lead us to weight the errors in our criteria 
for an optimal estimate. We once again look at the expected power spectrum of the 
noise: 

I rri , A _ Ffj.l\ I ® 

(s.4e) 




We note that the expected power spectrum of the noise is 

m*S nt , + S n 


S t = 




(5.47) 


m 3 +1 

and once again we should weight the errors, Since the denominator in the above 
expression for the power Epectrum is a constant, it can be ignored in the weighting, 
Thc: estimate fti should minimize 

““ m 3s's„. 


m 3 Hh 1 

This is equivalent to prEflltering the data with the frequency domain filter 


(5,4$) 


(5,49) 


V ''m*5 n . + S ni 

We note that our data filter we use to estimate rh depends on the true value of 
the unknown parameter., m, We therefore art forced either to adopt an iterative es= 
timation procedure, or to simply approximate our filters to the theoretically optima" 
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filters. We chose the second course,, as it is not clear we will know the noise spectra, 
exactly enough to know the true data filters to the accuracy where approximating 
m would make a difference. 

Handling Missing Measurements 

When there are data that appear in. the parameter estimation equations but are 
not measured directly we must Generate that missing data. This requires that we 
measure some other data that is directly related to the missing data by some known 
transformation, Wc apply that known transformation to the measured data to es¬ 
timate the missing data,, and we also need to transform the noise power spectrum 
of the measured data to generate the noise power spectrum of the missing data, Tn 
this example we reconstruct acceleration from velocity measurements. 


Conclnsinns On Our Estimation Procedure 

1, When there is substantial noise on more than one source of data it is appro¬ 
priate to use an eigenvalue/eigenvector based parameter estimation algorithm 

2, The optimal filter (in the frequency domain) for the prototype problem is given 
by 

, — ..... (5.50) 

3, To design the optimal filter we need to know noise characteristics such as the 
power spectra and S „ t . 

4, The true optimal filter depends on the true Values of the parameters Co be 
es Lima Led, We are Forced either to iterate or Co approximate, and we choose 
approximation in this chapter, 

5, This estimation approach can easily be generalized to apply to more complex 
problems with more than two measurement*. 
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6. Missing data and its noise spectrum should be reconstructed from other mea¬ 


sured data if possible. 


5,3 A Filter Design Example 

Figure 5-4 sbowe an example parameter estimation problem and the resulting optimal 
filters. 

We present a simple example to clarify the concepts presented in the previous sec¬ 
tions. Force and velocity are measured as in the prototypical problem* and the noise 
on each is independent zero mean white gaussian noise with variance 1. Therefore 
their power spectra are simply constants. We note that differentiating the velocity 
to estimate the acceleration will result in an acceleration noise spectrum equal to 
‘A The true value of the muse parameter is assumed to he 1 in this example* so the 
frequency domain data filter is 

I 1 

— ■ T 5.51 

Vo? 1 + 1 


W&. + S. 


V rM M n* 

The frequency domain force filter is 




7^Tt < 5 - M > 

while the frequency domain velocity filter combines a differentiating filter with the 
data filter. The magnitude of the velocity filter is 

u 


fiirirT (S.fia) 

V w 3 +> 1 1 f 

and the pha&a of the filter is a 90* phase advance at all frequencies* due to the 
differentiator. 

t,Et us examine what happens to the filters as the frequency, w f goes to ®ero and 
infinity. As the frequency goes to 0, the force filter becomes 

I 


lim - - 

--*<> y'w 3 i 


{5.54} 
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Figure S-4i Eswnpt* fikere, {Maw * 1) 






















and the velocity fitter becomes 


,, w 

]im — - -* lu 1 

U—*0 * a|* 3. 


(5.55) 


In the frequency domain, What is happening le at low frequencies the optimal filters 
essentially just differentiate velocity, and do nothing to the force signal. Ah the 
frequency gone to oo. the farce filter be conics 


tim 

W—+QO 


1 

v^+T 


i 

u 


(5.5d) 


and the velocity fitter becomes 


lira 

U «LXi 


yV -1 


l 


(5.5T) 


in the frequency domain. What is happening here is at high frequencies the optimal 
filters essentially do nothing to the velocity signal, and integrate the force signal, 

We see from the above limits and from the magnitude plots of the optimal fil¬ 
ters In Figure 5,4 that the optima) filters avoid differentiating the velocity at high 
frequencies, where noise would be greatly amplified,, but they also avoid integrating 
the force at low frequEncies, where bias and drift would be greatly amplified {.Figure 
5,5), 

The details of the filters depend oil the noise Epectra and the a priori estimates 
of the true values of the unknown parameters, but the fitters make intuitive sense 
in what they are trying to do. We note that getting the acceleration noise to zero 
tells us that the data filter should be just as the standard least squares approach 
predicts, 


5,4 Characterizing The Noise 

We ses that in order to design optimal filters for the data we need tp know the power 
spectra of the noise. Given that in real life one never knows the true value of the 
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Figure 5-5; Example Filters Compared, 







sign a!, but can only look at a noisy meuurenUGnt, It actnla unrealistic to expect to 
be able to find the noise power spectra- However, with no more senEing than the 
sensing we have already postulated, we can estimate the noise spectra of the data. 
We simply make use of the redundancy in ihe sensing, 

Let us assume we measure force and velocity 

/M-/<0 + »/M (s.&a) 

v(t) = vftjl + *w(t) [fi.SQ) 

We will let the true value of the mass of the object be 1 for this discussion, so that 
f{t) = ~iilY), although it is quite easy to derive the same results far the general 
case. We assume that fty(t) and n^ft) are zero mean, stationary, and independent. 
Let us examine what the power spectra of the observed data will be. First we 
note that the transfer function from velocity to force is 

3 = N{s) ($.60) 

The power spectrum of the true force signal will be related to the power spectra of 


the true velocity signal by 

(5.6l) 

and the true cross spectra will be 

- lTU»)Si M = (*.«) 

The spectra and cross spectra of the actual data can now be easily founds 

5,(“) =«/(“) +M") (S.S3) 

*,(«)=*,(«) + *»(») (5.54) 

s vj (w) = n(^)$i(u) = ( S6S ) 
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This provides us with enough information to estimate the noise spectra: 


Sn,(w) - S,(u) - Su f {v)H A [ji») 


(5.66) 



[5.67} 


We have implemented this approach to noise character nation on simulated data 



noise. 


Conclusions On Noise Characterization 

1. W# can use “redundant' 1 sensing to characterize noise spectra. 

2. We need to know the true underlying relationship between the redundant sig¬ 
nals, In this case we needed to know: 

* The true mass m r 


* aw - im 


3. Tlik approach to noise characterization generalizes to many other redundant 
sen Finn; situ at ions. 


5.5 Discussion 

One may ask whether the? additional complexity In data processing suggested by this 


chapter is worth the effort. We are in the process of applying this scheme to a real 
parameter estimation problem with data from actual hardware. This will enable us 


to more concretely assess the procedure's benefits. 

We should point out, however* that one makes data filtering decisions a[| the 
time. If ive numerically differentiate data we also low pass filter it, and must choose 
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the characteristics of the filter. If we integrate data, we must choose integration 
intervale, which Eet the filtering characteristics of the integrator. We should at least 
be aware of what good filtering decisions Look like, even though we may not apply 
the optimal filters directly. 

We can ask whether the filtering that is suggested here makes intuitive sense. 
Clearly, the noLEe matters in designing a filter. If there is almost no noise In a certain 
frequency range we should pay attention to those frequencies, while if the noise is 
much Larger than any signal at Soine frequencies we should probably ignore those 
frequencies. This is exactly what the filter tells us to do, Also, the decision of when 
to integrate versus when to differentiate; data seems reasonable, as explained in the 
filter design example section. 

We are solving a nonlinear estimation problem, and Date would have expected an 
iterative process somewhere. Note that the computation of eigenvalues and eigen¬ 
vectors, or singular values and singular vectors require an iterative algorithm, and 
this is where the iteration occurs. Such eigenvalue/singular value computation pro¬ 
cedures are extremely well developed and a lot of effort has been put in to making 
them numerically stable [Golub and Van Loan, 10S3), 

5*6 Conchieiom 

We have developed a parameter estimation procedure that shows us appropriate 
ways to filter data for parameter estimation. The procedure addresses two main 
problems: 

1, Noise is typically present in all data. 

2, Often we must reconstruct miE&ing measurements. 
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5,7 Appendix 

5.7.1 Deriving The Dynamics Equations 


The equations for the forces and torques (expressed in the load coordinate system) 
necessary to move a load along a given trajectory are 

f = mp - mg + W X me + tit x [v x me) {E.6S) 

11 = Tih + w x (Iur) + me x p - me x g ( 5 ,^ 9 ] 

where: 


f = the net force on the load,, 
m = the mass of the load* 
p = the acceleration of the load, 

S = the gravi £y vector (g = [ 0 ,G, - 9,8 meters/sec 8 1 ) p 
ur — the angular velocity vector, 
w = the angular acceleration vector, 

c — the unknown location of the center of mass of the load, 
relative to the force sensing coordinate system origin, 

11 = the net torque on the toad, and 
I — the moment of inertia tensor of the load about the 
force sensing coordinate system, origin. 

In order to formulate the above equations as a system of linear equations, the 
following notation is used; 





f 1 


O 

3 

JF 

£ 



u X n = 

LVj U -Wj 


C„ 


— l*? r (L^. () 


.. 



(5.70) 


103 






where 
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(5.7 L) 


(5.73) 


Using these expressions, Equations (5.68) and (5.65) can be written as a single matrix 
equation expressed in load coordinate frame. This load dynamics formulation is 
presented sn Equation (5.12), 


6.7.2 Integrating The Forces And Torquea In The Force Sen¬ 
sor Frame 

Throughout this derivation we will use the superscript notation a and p to indicate 
what coordinate frame a vector is expresaed in, if there ia any confusion. If there U 
no such superscript, the vector is expressed in the load frame. 

The term P ie integrated using equation 7.22 of (SymDn, 1571): 

H^("p) = *W + R('u X *p) (3.73) 

were E ie the rotation matrix representing the rotation from an inertiai coordinate 
ayatem to the force sensing coordinate system that continuously moves with the load. 
To get an integral form of this equation we simply integrate It: 

ft+T 

/ R-*pdr =(»■**) 


l 


p+j 


Rf’wx *p)dr 


(5.74) 
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and performing the indicated rotations leaves us with 



i+T 

+ 

i 



{5,75) 


Similarly 


/ t+T 

p ij4r 


I P+T rt+T 

+ / X 

! h 


F ti/dv — P i/t 


t+r 


since w X w = 0. We also remember that 


rl+T / fi+T \ 

I, ’** = (( **)*■ 


(5.76) 


[5.77} 


Each of the matrices [ F tLix| and ’*cj can bo integrated element by element to 
show that 



[5.78) 

(579) 


The matrix i'(g- pjxj can be integrated element by element in the Same way. Each 
matrix element of the terms [ p ti#x|[ p itfx| and pwxj^u] a re numerically integrated: 
by adding values at each time step. The result of this integration Is given in Equa¬ 
tion (5.M). 
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Chapter 6 


Single Trajectory Learning 


6*1 Abstract 

We present an algorithm enabling a. robot that Is repeating the game trajectory to 
reduce its trajectory error an repetition of the motion. The algorithm details, how 
to use the trajectory following error (the time history of the differences between the 
desired trajectory and the actually executed trajectory} after any particular move¬ 
ment to update an actuator feedforward command that is used for the subsequent 
attempt. This approach to robot learning ia based on explicit modeling of tha robot; 
and uses, an inverse of tha robot mode] as the learning operator which processes 
the trajectory errors. Results are presented from a successful implementation of this 
procedure on the MIT Serial Link Direct Drive Arm. The major point of this chapter 
is that more accurate robot models improve trajectory learning performance, and 
learning algorithms do not reduce the need for good models in robot control., 


J Thifl chapter in a reviled venjanof (AiJujcn and Melulyre., 




6,2 Introduction 


Adaptive ffiRdfnrivjirtl control for repetitive motions.: In actual use robots 
tend to execute the same sequence of motions with the aaroe loads repeatedly. We 
can take advantage of this pattern of usage by specializing the robot control system 
to store feedforward commands in a memory and play them back when necessary. 
This type of control system would repeat its errors on each movement* in contrast 
to the performance improvement with practice seen in human movement control. 
We propose an algorithm that uses pracLice to improve movement execution, by 
altering the stored feedforward commands cm the basis of previous movement errors. 
This serves two purposes: l) to improve robot trajectory following for repetitive 
movements* and 2) to Increase our understanding of the role of practice In human 
motor control. 

G.2.1 Previous Work 

Recent work in a number of laboratories has focused OJ 1 how to refine feedforward 
commands for repetitive movements on the basis of previous movement errors. The 
first paper on repeated trajectory learning Stems to have been (Uchlyama, 1978}. 
Subsequent work includes (Arimoto, 1985; Aiimoto et, al., 1954a, 1954b, 1984c, 
1985; Hara, Gmata, and Nak&no, 1955; Kawamara et, a],, 1951, 1955; Casalsno 
and Gambarddla, ]98fi; Craig, 1984, 1983; Turula and YamakiLa, 1986; Harokopos, 
1986a, 1986b; Mita and Kato, 1985; Merita, 1986; Tngai and Yamano, 1955, 1956; 
Wang, 1.984; Wang and Horowitz, 1985) 

These papers dtSCU39 using linear learning operators and have emphasized stabil¬ 
ity of the proposed algorithms. There has been little work emphasising performance* 
i.e. the convergence rate oF the algorithm. Simulations of several of these algorithms 
have revealed very slow convergence and large sensitivity to disturbances and sensor 
and actuator noise, 
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6,2*2 Features Of This Work 


What distinguishes this trajectory learning algorithm from previous robot trajectory 
learning schemes is the following combination: 

Provides guidance for designing the learning operator; The central prob¬ 
lem In making use of trajectory error measurements is transforming those errors into 
feedforward command corrections. Previous work has explored the requirements on 
the learning operator for convergence, but has given little guidance as to how one 
should choose a particular Learning operator from the large set of learning operators 
that converge. The algorithm proposed here explicitly uses an inverse plant model 
as the learning operator. If there are no disturbances or sensor or actuator noise 
and we have perfect models of the robot, the algorithm will correct any errors in the 
feedforward command after one practice motion. 

Handles nonlinear plants: The algorithm can make use of the nonlinear rigid 
body robot dynamics In the model used to generate feedforward command correc¬ 
tions. Previous work have used linearized or otherwise simplified robot models. 

Makes explicit use of feedback control: The algorithm takes Explicit ad¬ 
vantage of the on-lins trajectory improvement provided by the feedback controller 
in calculating the next feedforward command. 

Successful implementation: The algorithm has been implemented on an ac¬ 
tual robot, the MIT Serial Link Direct Drive Arm, Excellent trajectory learning 
performance requiring only' a small number of practice trials has been achieved. 

Generality; works with many feedback controllers: The algorithm ap¬ 
plies to a wide variety of feedback controller structures, as only knowledge of the 
feedback controller output is required, ft would be easy to combine this adaptive 
feedforward control algorithm with adaptive feedback controllers. 

Generality: works with many plants: The algorithm does not require the 
plant dynamics to be of a particular type and applies to a wide range of robots 
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with (for example): actuator dynamics, joint compliance dynamics, arid flexible link 
dynamics. This is an important feature of the algorithm, as it Is becoming dear that 
a typical robot exhibits quite complex dynamics (Sweet and Good, 1985}. 

Analysis relevant to other schemes: The analysis of this algorithm makes 
clear why other trajectory improvement schemes that intuitively seem correct often 
perform badly or actually degrade performance in practice. 


6.3 The Trajectory Learning Algorithm 

6.3.1 The Control Problem 

Our goal is to repetitively execute an unrestrained robot trajectory with as small 

а, trajectory following error as possible. The desired trajectory has a finite time 
duration and the robot starts in a known initial steady state,, x(0). We assume 
that modelling errors are much larger than any sensor and actuator noise or plant 
disturbances, and Save the question of how to appropriately filter out non-repeat able 
noise and disturbances for a later paper. 

There are three components of the trajectory Learning algorithm: feedforward 
command initialization, movement execution, and feedforward command modifica¬ 
tion (Figure 6.1). After initializing the feedforward command,, the movement exO’ 
cuLion and feedforward command modification steps are executed for each repeated 
movement attempt. 

б. 3x2 Fetid forward Command Initialization 

Given the desired s-taLc trajectory specification, xj(£), the feedforward command 
memory is initialized using a model of the inverse of the robot dynamics (Fig¬ 
ure 6.1 A}: 

=^" 1 (x, i (f) s x J (f).J (6 1) 
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A: feedforward Command Initialisation 



Dr Movement Execution 



C: Feedforward Command Modification 



Figure 6,1: Block diagrams for feedforward command Initialization, execution! and 
modification. 
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Using Inverse models of robots to generate feedforward control signals: 
The robot itself transforms a vector of actuator commands, u, into motion., x 

x = if(x, uj (6,2) 

x represents the state vector of the robot, which includes the position and velocity 
of each joint, x ia the derivative of x. A model of the inverse of the robot dynamics 
transforms a trajectory specification, x(tj, into actuator commands necessary to 
achieve the desired motion: 

ti=A -l (x,x) (6.3) 

For robot arms a rigid body dynamics model is often used to predict the forces and 
torques necessary to achieve a particular motion, and thus serves as a anode! of the 
inverted robot dynamics. The rigid body dynamics equations for a robot can be 
written as 

^(x,*) = Torques = 1(0) ■ 6 + $ - C(ff) ■ 0 + g(fl') (6,4) 

where fl(f) is the desired trajectory of the joint angles, 1 ( 0 ) is tbs inertia matrix of the 
arm, Chi?) is the Coriolis and centripetal force tensor, and gi0} Ls the gravitational 
Force vector (Holler bach, 1084). For some robots it is argued that additional sources 
of dynamics are important (Goof, 1085a; Sweet and Good, 19S5; Good, Sweet, and 
Stroebel, 1965-), In these cases we can still model the rohot dynamics and Invert the 
model. For the purposes of this chapter we will uac only rigid body robot models, 
as these types of models describe moot of the dynamics seen in a direct drive robot 
arm. 

Generating the robot model: In order to use a rigid body robot model the iner¬ 
tial parameters of the robot must be known. Thera are several sources of parameter 
estimates: l) CAD/CAM models may be used tO' compute inertial parameter esti¬ 
mates, 2) the robot can be disassembled and the inertial parameters of the parts 
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measured* or 3} the inertial parameters can be estimated using movement and actua¬ 
tor data from actual use of the robot. In previous work we have taken the parameter 
estimation approach and shown that the dynamics are linear in the unknown iner¬ 
tial parameters of the links, We have identified the rigid body dynamics of the MIT 
Serial Link Direct Drive Robot Arm, and have used this dynamic model for control 
(An, Atkeaon, and Hoilerbach, 198&), 

6.3.3 Movement Execution: 

The i'tb attempt at the desired movement is executed using the current feedforward 
command, [Figure 6.1B). This command includes an error, which is 

reduced by the actions of a suitably designed feedback controlled C7, leaving an error., 
in the actuator ■commands sent: to the robot* u’(t). For a linear plant and 
controller the amount of error reduction can be expressed using Laplace transforms; 

u ' w = itcim ^ 11 

The actuator commands and the actually executed movement, trajectory* ix^tj* are 
stored for use by the learning module. 

0,3,4 Feedforward Command Modification 

The previous steps in this algorithm are standard components of many current robot 
controllers. One contribution of this work is to propose a particular approach to 
transforming trajectory errors Into modifications of the feedforward command. We 
represent errors in our dynamic models of the robot as an input command distur¬ 
bance, which we can correct for by modifying the feedforward command. We use 
our models of the inverted robot dynamics to estimate this actuator command error, 
which is then subtracted from the previously applied actuator commands to 
form the feedforward command for the next movement attempt (Figure 6.1C). 
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Estimating the actuator command error 

We isfluma there exists ft- correct actuator command history, u*(f) T that drives the 
plant exactly along the desired trajectory in the absence of disturbances or noise, 
and define the current actuator command error as 


Su'(t) = u'(l) - u-(0 = «-'(*'(l),**W) - (6.B) 

where R T Q lathe true inverse robot dynamic*. Our estimate of the error in the cur¬ 
rent actuator command ig found by simply replacing the true LilVcrse robot dynamics 
wi th our model: 


MO = 


(fi.T) 


When the plant is linear we can apply the plant inverse directly to the trajectory 
error. 

R~ 1 fx, x) - j& _1 (xj, xj) = iJ " * (e, e) (6.8) 

The command error estimate can l>e expressed in terms of the feedforward command 
error using Laplace transforms. 


pi « = r+$Pw^" w 

R(,) 


( 6 . 9 ) 


s ”« - Sw c ' W = W)TTctim e “>M «*“» 

Updating the feedforward command 

The update for the feedforward command on the next movement is simply the mod¬ 
ified actuator command. 

(eu) 
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By subtracting the correct command, u*, from both tides of Equation (5-11) and 
using EquatsoriA (6,5) and [6,10) we can express the Laplace transformed error prop¬ 
agation equation for the linear can: 



fl(«) ~ fl(«) c i { s 

+c{*)R(*)y n{ 




With a perfect model, ii = J2„ the feedforward command errors wilt be aero after 


one movement. 


6*4 An Implementation Of The Algorithm 

We have implemented the trajectory learning algorithm on the MIT Serial Link 
Direct Drive Arm. This three joint arm la described in (An, Atkeson, andHollerbach, 
1965), To explore the effectiveness of our learning algorithm we wj]] present results 
on learning a particular trajectory. 

Discrete vs. Continuous Time: Since we are using a computer to generate 
the control signals, all signals and controlled systems were represented in discrete 
time rather t ha n, continuous time. 

The Test Trajectory: All three joints of the Direct Drive Arm were commanded! 
to foliaw a fifth order polynomial trajectory with zero initiaL and Una] velocities and 
accelerations and a 1.5 second duration. Figure 6.2 shows the shape of the trajectory 
for each joint, and Table 6-1 gives the initial and final joint positions„ the peak joint 
velocities, and the peak joint accelerations. 

The Feedback Controller: An independent digital feedback controller was 
implemented far each joint and wm not modified during learning. 

Initialisation Of The Feedforward Command: The initial feedforward 
torques were generated from a rigid body dynamics model. The model and the 
estimation of its parameters are described in (An, Atkeson, and Hollerhach, 1985). 
The calculated feedforward torques are shown in Figure 6.3A. 
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Position 




Acceleration 



0.5 seconds 


Figure 6.2: Desired trajectory. 


Joint 

Initial 

Position 

eadtiFli 

Fina) 

P osition 
radians 

Peak 
Velocity 
radians f $ 

Peak 

Acceleration 

rodtaru/* 3 

1 

0,5 

4.3 

3.0 

±10,3 

2 

s.o 

1.0 

-5,0 

±10.3 

a 

4.0 

^0,5 

-5.6 

±11,5 


Table G.1: Test trajectory parameters. 
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Figure 6.3: Feedforward Torques, - ■**«(**) 
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Figure 6.4: Velocity Errors, Maw/iKwi) 












Initial Trajectory Performance j As an index of trajectory following perfor¬ 
mance the velocity errors (the difference between the actual joint velocity and the 
desired joint velocity) for the first: movement are shown in Figure &. 4A. We have plot¬ 
ted the raw velocity error data to give an idea of the relative size of the trajectory 
errors and sensor noise. 

Calculating Acceleration and Filtering? In order tD use the rigid body 
inverse dynamics model to compute joint torques it was necessary to compute the 
joint accelerations,. Joint positions and velocities, were measured directly. A digital 
differentiating filter combined with an 8TI3 t low pass fitter was applied to the Velocity 
data to estimate accelerations. 

To reject noise and noo-repeatabUe disturbances it is necessary to fiiter the trajec¬ 
tory errors and controller output to improve the convergence of the learning process. 
In this implementation we applied low pass digital filters with an 8Hz. cutoff to the 
data used in the learning process. We filtered the references used by the learning 
operator with the same filter used on the data.. It was necessary to correct for tncon- 
sbteneiea between the velocity eenBors and the position measurements N which was 
done by calibrating the position reference to tha feedback controller* 

Final Trajectory' Performance: The robot executed two additional training 
movements which are not shown, and ita performance on the fourth attempt 0 : the 
test trajectory was assessed. Figured,3B shows the modified feedforward commands 
used on the fourth movement, and should be compared with the predicted torques 
shown in Figure 6.3A. Figure 6.4D shows the velocity errors for the fourth movement, 
and should be compared with the initial movement velocity error e in Figure 6 . 4 A. 
There has been a substantial reduction in trajectory following error after only three 
practice movements. 
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6.5 Using Simplified Models 


It may seem unnecessary to use the full rigid body dynamics model of the robot in the 
learning algorithm. One might think that learning algorithms of this type allow one 
to avoid modeling the robot in Full detail, This is not the case. Simplifying the robot, 
model necessarily introduces additional modeling error, Without careful analysis, 
such modeling errors may cause the learning algorithm to have poor performance, 
or even to degrade performance. As an illustration of the possible effects of modeling 
error dne to the use of simplified models, wc will now present a seemingly reasonable 
simplified model of a two joint robot arm that when used as the learning operator 
fails to Learn. The robot arm lb a planar two link mechanism with rotary joints and 
is described In the introduction to (Brady, ct aL, 1982.) 

The simplified dynamic model that we have chosen for this example is that of 
two independent rotary joints with constant moments of inertia. That is, we ignore 
the centripetal and coriolis torques of the complete rigid body robot dynamics, and 
wo assume constant moments of inertia around each joint. The moment of inertia 
of link 2 is a constant with respect to ff a . The moment of inertia around joint 1 
depends on 0j. We approximate the moment around joint I as the average oF the 
maximum and minimum moments around that joint, over all possible configurations 
oF the robot. The equations of motion far such a simplified system are: 

Torques ss I - 0 ((J,L3) 

where I is a constant diagonal 2x2 matrix. This gives a learning operation of: 

ujy = U 1 ' - I«tLn»lcd ■ (0 - h] (0-14) 

The results of applying this learning algorithm to the simulated two joint robot 
movement arc show in Figure 6,5, The movement is from point a to point b with zero 
initial and final velocity, acceleration, and jerk (seventh order polynomial), Feed¬ 
back control is provided by independent PD controllers at each joint, each having 
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Figure 6.5: Simulated Two Joint Learning With Decoupled. Mode! 
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4 bandwidth of 1.0 hertz and damping coefficient of 0,707 {based on 
Figure 6.5 A ah owe the performance of the system on the initial trial, with the ini¬ 
tial feedforward torques, U^, based on the simplified dynamics model. Figure 6.5B 
shows the performance of the system after five Iterations of the Learning algorithm. 
In this case, where an inaccurate Inverse model of the robot has been used to update 
the feedforward torque command, no improvement- in trajectory Fallowing perfor¬ 
mance is seen. Had the fuEl inverse dynamics model been utilized (under these ideal 
conditions of no measurement noise, actuator noise, or external torque disturbances), 
our algorithm would have produced a perfect movement after one iteration. 

Appropriate .Robot Models: It has been argued that simplified models are 
appropriate for a robot with high gear ration such as the PUMA. One must still mode! 
the other sources of dynamics prominent in these types of robots [Sweet and Good, 
lt>65; Good, Sweet, and SLroebel, iQSJi). Higher order actuator dynamics may play 
an important role [see, for example, door [198oa,b)}. Our paint is not that the rigid 
body dynamics are the only appropriate model and must be used, but that we must 
be careful! to include all significant dynamics in our models. Learning performance 
can be used to assess the quality of the models used to drive the learning. 


6.6 Convergence 

An important question that arises is how dose our model of the inverted robot, dy¬ 
namics has to he to the true inverted robot dynamics for the proposed trajectory 
learning algorithm to converge to zero trajectory error. We will indicate: how conver¬ 
gence can be tested in the general nonlinear rase, but by specializing the convergence 
criteria to the linear case and presenting a numerical example we will show that a 
convergence proof does not guarantee acceptable performance. 
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6.6.1 Nonlinear Convergence Criteria 

Analogies to solving simultaneous nonlinear equations: One way to view 
the proposed trajectory 1 Earning algorithm is as a procedure to solve the vector 
nonlinear equations: 

Xt = Rintf) (6.15) 

for ji//, where a the sampled version of the desired trajectory expressed as a vector 
(xj ji],Sld[2],,. .sX^T]} 7 ’, M// is a similar Vector of the feedforward commands,, T is 
the number of samples in the trajectory, and £ is a nonlinear function representing 
the true robot and feedback control ler dynamics. Since the initial state of the robot 
at the beginning of each movement is a known constant, J5{] does not require the 
robot, initial state as an explicit argument, 

Testing for convergence using fixed point theory: One approach Id analysing 
convergence of algorithms to solve nonlinear equations is to combine the original 
equations and the proposed algorithm into a angle iteration function. Each cycle 
of movement repetition and Feedforward command modification, can be expressed as 
an iterative function, 

= SW,) (6.1«) 

and we can appeal to fixed point theory for convergence conditions for fijj [Isaacson 
and Keller, 1966). This is the approach suggested by {Wang, 1.984; Wang and 
Horowitz, 1985). 

6.6.2 Convergence Does Not Guarantee Good Performance, 

Although conceptually elegant, the convergence conditions provided by fixed point 
theory arc too weak to be useful, The reason for this is that the duration of the exe¬ 
cuted trajectory Is finite and the controlled plant is causal. The trajectory duration 


is referred to ag the learning interval, -as thig is the Unit over which the feedfor¬ 
ward command Is modified. Convergence can he guaranteed hy assuring that the 
feedforward command errors propagate Forward in time Oil each cycle of movement 
execution and feedforward coprmrifl-nd modification until the command errors propa¬ 
gate out of the learning interval (Figure 6.6}. Command and trajectory errors can 
grow exponentially as long as they are continually shifted Forward in time, as we will 
show in the following example. 

The example plant 

Consider a rotary single degree of freedom robot. The dynamites of this robot in 
continuous Lime are 

r = IS (6.17) 

where r is the torque at the joint, I is the moment of Inertia of the robot, and B is 
the angular acceleration. We can express the dynamics of this robot in discrete time 
as 

h ! 

tik — 20Jfc-i ■+■ Bk-i = 2/( r * + T *-i) (O.lfSj 

where h is the sampling period (Astrorn and Wittenmark, 19-54), Note that for 
later iiotitiona! convenience in inverting the plant the sampled torques have been 
renumbered to remove the plant delay, A sampling frequency of 1kHz is used in the 
numerical simulations to follow. 

We use a feedback controller 

= -H a “ -c.)- - «*) («»} 

to give the second order plant a natural frequency of lOfte and a damping ratio 
of 0.707. The angle and the angular vElocity of the single joint are both measured 
directly. 
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Deriving the convergence criteria 


Since the plant is linear the inverse plant model used for feedforward command 
modification will he linear„ and it can therefore operate directly on the trajectory 
error,, 6 i|. The example plant is also minimum phase and the plant inverse will he 
causal. In order to put convergence hounds on the learning operator we will refer 
to a general learning operator L instead of requiring the learning operator to be 
the Inverse of the robot dynamics, .ft -1 . L will be restricted to be a causa, linear 
operator. 

The propagation of feedforward command errors from one movement to the next 
is given by subtracting the correct command, u*, from both sides of Equation (6.11) 1 




[ 6 . 20 ) 


= diT[f| - i|f] *■ e[t] 

where ij(] ia the Impulse response of the learning operator L and + is the convex 
lution operator. Since the learning interval (the duration over which we modify a 
particular feedforward command) k Finite and includes only T samples, we can ex* 
press Equation (6.20) as a matrix equation by replacing convolutions with matrix 
operations. 

The sequence of sampled command errors and feedforward command errors are 
expressed as vectors, and the transformation between them can be expressed as a 
matrix equation Jii' = Afiu'fp 


f M°1 1 

/ 

Mi] 

= ' 

1 MU ; 

L 


c[0] 0 

a[l] a[ 0 ] 


0 

0 


&[t>] j 


\ 




} 

l J 


( 6 . 21 ) 


where A is a lower triangular Toephtz matrix. The elements n f[ are the impulse 
response dF the transfer function relating feedforward command errors to actuator 
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command errors* f/[l + C[a]JI(f]), *n.d wc note that a 0| must at ways be 1 due to 
& minimum loop delay of one sample before the feedback command ran compensate 
far feedforward command errors. 


The trajectory errors are generated by feedforward command errors according to 


the matrix equation o' = Bdll^ r : 


( e*[0] 1 


^ Bio) 0 ■ ■ ■ 0 





ti[fj b[Q] — 0 

“ ■ _ r 

a j + ■ 



L J 


■ B T r 

,*P1 t|T-l| ■■■ i[D]j 


t HAn j 


( 6 . 22 ) 


where fc|i| is the impulse response corresponding to the transfer function R [jsj/[l +■ 

The effect of the learning operator can be expressed as a lower triangular Tocpllti 
matrix f L, vising the matrix equation fin’ = Le ; : 


r fdV] i 


f J(0] 0 ■ ■ ■ o 1 


' e* |0j 1 


= 

f[l] J[0] ■ - 0 

•K m J 1 

■r ■ ■ + 


^[1] 

L $u[T] j 


r , 

'll] i[T - 1| - i|0| j 


v <n j 


(6.23] 


Combining the previous matrix equations. Equation 6.20 can be written els Lhe 
matrix equation: 

fiujy = (A - LB)da^ (6.24] 


Testing for convergence of the feedforward command error to Kero reduces to 
guaranteeing that the absolute values of all the eigenvalues of the matrix (A - LB) 
are less than I. Since A, L, and B are at] lower triangular Toeplits matrices* the 
eigen Via] ues arc all equal to the d lagonaE element oF (A - LB) t a jOj - l [0| b |0| - We rscal | 
that, a10 — i* and note that the tost for convergence in the Unite learning interval 
case reduces to a test involving only the first element of the learning operator impulse 
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reap bn &e and the controlled plant impulse respond; 


fl = if0|&|0i]| < X 


[ 6 . 25 ) 


This convergence teat does not require L to be a very accurate inverse plant model- 


A numerical example 

Ta demonstrate that the above convergence condition does not guarantee accept¬ 
able performance let us example a particular learning operator which satisfies the 
convergence test but generates bad performance. Let us choose a learning operator 
that satisfies the finite learning interval convergence test exactly such that 


l - i[0|ft{0] = 0 < 1 


[ 6 , 26 ] 


Such a learning operator is given by 




0 otherwise 


[ 6 . 27 ) 


With this learning operator the eigenvalues of {A -- LB) arc all zero, and we are 
guaranteed exact convergence in at most T movements, where T is the number of 
simples in the learning interval This learning operator corresponds to correcting 
the feedforward command with a scaled version of the position error history. 

To examine the transient response of this learning operator we resort to nu¬ 
merical simulation. We initialize the feedforward command to have a single error 
on the first sampler The resulting position error is shown on the first graph of 
Figure 6.6. Several of the following movements are also shown, and by the 10th 
movement the position error has increased by a factor of IQ® 4 * It is difficult to see 
that with, each movement the first non-zero position error of the previous movement 
is exactly cancelled, as the position errors later during the movement are blowing up 
exponentially. This exponentially growing wave of errors does shift forward in time. 
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1.0 POSITION ERROR 





Figure 6,6-: Simulated learning performanK for a bad plant inverse mo6t\ 
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and after T movements will have shifted completely outside the learning interval. 
Had the correct Inverse dynamics model been used as the learning operator [under 
these ideal conditions of no measurement noise, actuator noise, or disturbances) the 
feedforward command error wouEd have been entirely cancelled after Due movement. 
This type of finite time interval convergence relies on. the absence of any intro¬ 
duction of new errors. In the presence of sensor and actuator noise and plant distur¬ 
bances the convergence promised by the finite time interval analysis will probably not 
occur. In addition, as the campling interval length changes in the discretization of a 
continuous time plant, the convergence properties of this type of learning algorithm 
■change. Convergence is improved when the plant is sampled at low frequencies. 


A performance test 


The previous example demcmBtcates the need to test performance in addition to 
showing convergence of a learning algorithm, in she nonlinear case performance 
must usually be checked by simulation and. by actual implementation, but with 
linear plants and feedback controllers we can take advantage of superposition and 
the ability to transform to the frequency domain to check how errors at all frequencies 
are affected by the learning algorithm- From Equation 6,12 or 6-2Q wc ace that the 
errors are propagated according to 






{A. 28 } 


where the argument z indicates that we have shifted to the frequency domain using 
the 2 transform. With a perfect inverse model, [L = i?" 1 ), A [a] is zero, implying 
convergence of the feedforward command error to zero after one practice movement, 
The learning algorithm will perform badly if at any frequency the gain of A|aj is 
greater than 1. The gain of A.*] as a function of frequency can be used to evaluate 
the efficacy of proposed learning operators, in the numerical example showing con- 


128 




vcrgcince with poor performance, Lbe poor performance Wis indicated by a maximum 
gain of Aj'i] equal to 530. Thin is approximately the factor by which the maximum 
position error was growing on each movement repetition in the simulation. 

6.7 Concilia ions 

The contributions of this work arc: 

* The derivation of a nonlinear trajectory learning algorithm based on explicit 
[noddling of the plant, 

* The demonstration dF good performance of the 1 Earning algorithm by imple¬ 
mentation on the MIT Serial Link Direct Drive Arm. 

■ The demonstration by a simple example that proofs of convergence do noL 
guarantee acceptable performance and of the need to Verify performance. 

We conclude that more accurate robot models improve trajectory learning perfor¬ 
mance, and learning algorithms do not reduce the need for good models in robot 
control. 


Chapter 7 


Future Research 


There are many components of motor learning* and the work in this thesis is only 
a Email step in a research program in motor learning. Some of the future research 
which is very closely related to the research presented in the thesis is discussed in 
the previous chapters- Other topics are mentioned here. 


7*1 Local models 

The single trajectory learning work can be extended to learning a group of similar 
trajectories. TIlia Involves buiSding a representation of the dynamics in a small region 
of the augmented state space (the state variablcE and their derivatives], which I refer 
to as a local model. The local model could be a representation of the linearized 
dynamics about one of the desired trajectories in the group 1 or it could be a tabular 
representation. This local modelling would be a correction model to the previously 
identified global model. Key issues here are storage of ail the Information necessary 
to represent the fine details of the local dynamics, and deciding whether to pool 
Information from different taska or Etore experience according to the type of task. 
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7,2 Task level learning 


Thts thesis addressed only Isolated learning modules. It is important to examine 
learning a complete task in addition to focussing ou modules, its there will be many 
learning modules acting simultaneously in any realistic motor problem, and exploring 
their Interaction is important. It is important to focus on other kinds of tasks where 
the effector system must respond to external demands rather than follow a given 
plan. 

Often task performance can be represented by a Email number of parameters, 
and the execution of an arbitrary trajectory" is not required. Reducing the degrees 
of freedom of possible motor performance greatly simplifies the learning problem, 
and reduces the amount of knowledge necessary to learn from practice, 

7.3 Plan optimization 

The work in this thesis has focussed on improving execution of a given plan. The 
perfect execution of a Hawed plan will not lead to optimal performance. Plans can 
be improved on the basis of models derived from experience, 

7.4 Object perception 

Rigid body Load identification is a first step towards general object perception on 
the basis of force/torque information. There are many types of loads that are not 
well modelled by the rigid body dynamics framework, and alternative approaches to 
identification and control Of these loads should be explored, 
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7*5 Human motor learning psychophysics 


This writ suggest* a program of research on the quantitative psychophysics of motor 
learning. My previous research, not discussed in this thesis, cm biological movement 
kinematics, dynamics, and control provides a strung base fur planned Studies on 
biological motor learning. Findings relevant to motor learning include an invariance 
of the tangential velocity profile across many different movement conditions, sepa¬ 
rability and scaling of load dynamics, and scaling of control gains with movement 
speed, fu the immediate future my biological research will focus on two areas: 

A Competence model: My own and other theoretical work have demonstrated 
possible roles of internal dynamic models in control and learning, and provided 
mathematical bounds on the required accuracy of these models. The goal of this 
research is to estimate, using simulation*, how good human internal models actually 
are. However, in order for these estimates to be meaningful we need to have accurate 
estimates of the mechanical properties of the arm and the effective feedback gains 
during movement, I have begun this research by examining elbow movement, and 
discovered both modulation of effective feedback gains during movement and scaling 
of effective feedback gains for movements made at different speeds. These measure¬ 
ments need to be extended to multi-joint movement, and then useful simulations can 
be undertaken. 

Motor Uarning psychophysics: Theoretical analysis and implementation on robots 
of different trajectory learning algorithms has revealed distinctive patterns of tra¬ 
jectory modification for alternative algorithms. In order to test for the possible use 
oF any of theEe algorithms by humans we need to quantitatively meaaurE patterns of 
trajectory modification during learning of particular trajectories. 
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7.6 Concluding note 


This thesis provides a particular view on wliat Intelligence ia: identifying models 
of both the environment and the organism itself, and using the identified models 
to predict* plan, and control action. Attempting to build intelligent machines that 
achieve or surpass human levels of performance tests one understanding of these 
ideas, and how complete this view of intelligence really is. 


Appendix A: Using The Identified Arm Model 
For Control 
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Experimental Evaluation of feedforward and Computed Torque Control 

C,H. Afl, C,G- AtfcfiSijR, J.D. Griffiths,, and J.M. Holler bu-ch 

MIT Artifical Intelligence Laboratory 
Cambridge* Mass. 02139 USA 

Abstract. Trajectory tracking errors resulting from the application of various con¬ 
troller* have been experimentally determined on the MIT Serial Link Direct Drive Arm, 
Tha controllers range from simple analog PD control applied independently at each 
joint to feedforward and computed torque method* incorporating fall dynamic*. It was 
found that trajectory tracking errors decreased as more dynamic compensation terms 
were incorporated. There was no significant difference in trajectory tracking perfor- 
mace between the feedforward controller using independent digital servos and the full 
computed torque controller. 

1 Introduction 

Despite voluminou* publications on the control of robot arms, there have heen few 
experimental evaluations of the performance of propoeed controllers. A major reason for 
this U the lack of a suitable manipulator for testing that fits these publications' modeling 
assumptions. Commercial robots are characterized by high gear ration substantial joint 
friction, and slow movement. As a result,, their dynamics are approximated well by 
single-joint dynamics (Goor, 19S&; Good, Sweet, and Strobe], 1985], Moreover, hardly 
any commercial robots allow the control of joint torque, which ia required in many of 
the proposed controllers. 

Direct drive arms are increasingly overcoming some of the performance limitations of 
highly geared robots (Asada and Yoncef-TDimfi, 1984; Curran and Mayer, 1985; Xuwa- 
harm et al. T 1985), The manipulator dynamics are made mo-re ideal by the reduction of 
joint friction and backlash effects, and the control of joint torques becomes more feasi¬ 
ble. Hence direct drive arms have the potential for serving as good experimental devices 
for testing advanced arm control strategies. When gearing is eliminated, however, the 
full nonlinear dyn ami c interactions between moving links are manifested. 

This paper reports on two seta of experiments with the MIT Serial Link Direct Drive 
Arm (SLDDA) involving a subset of proposed control strategies. The first set of ex¬ 
periments is based on a hybrid control system. There is an independent analog servo 
for each joint with the position and velocity refersneea, and feedforward commands 
generated by a microprocessor. Since most commercial arms are controlled by simple 
independent PID controller for each joint, an independent PD controller was tested on 
this arm to provide a baseline for comparison. The PD controller was augmented by 
feeding forward first gravity compensation and then the complete rigid body dynamics 
to ascertain any trajectory following improvements attained by taking the nonlinear 
dynamics more fully into account. The second set of experiments shows the preliminary 
results of digital servo implementation, using one Motorola 68000 based nucroproces- 



sor to control all the joint? of the SLDDA- The on-line computed torque approach is 
compared to the PD and to the feedforward approaches using the digital servo. 

The accuracy of the manipulator dynamic model impinges on the performance of 
feedforward and computed torque control. Slum friction is negligible for direct drive 
anna, and presuming that one has good control of joint torque*, the issue of accuracy 
reduces to how well the inertia) parameter* of the rigid Units are known, In our previous 
work, we developed an algorithm for estimating these inertial parameters for any multi- 
link robot as a result of movement,, and applied It to the SLDDA (An, Atkeson, and 
Holler bach, IMS), The present paper presents result? of utilizing the estimated model 
to control the robot by both off-line (feedforward) and on-line (computed torque] com¬ 
putation of the joint torque 

1,1 Control Algorithms 

The full dynamics of an fi degree-of-freedom manipulator are described by 

n = J{q)q + V(q p q) + Q{q) + F(q, q) (1] 

where n is the vector of joint torques (for rotational joints), q is the vector of joint 
angles, J is the inertia matrix, V is the vector of coriolis and centripetal term*, G ie 
the gravity vector, and F is a vector of friction terms. The simplest and moat common 
form of robot control is independent joint PD control, described by 

□ =X.(q*-q)+l£ r (q,-q) {2) 

where qj and qj are the desired joint velocities and positions, and IL P and K„ are n - n 
diagonal matrices of position and velocity gains. 

Feedforward control augments the basic PD controller by providing a set of nominal 
torque? tiff* 

tiff [ q s, qj, q j) = 3 (qj) qs + V( qs, <js) + G(q d ) + t (q*, q j) (3 j 

where the hat (*) refers to the modelled value*. When this equation is combined with 
(2), the feedforward controller results: 

H = a/j(q^ <3^ qj) + K,(<^ - A) + K f (q* - q) (4) 

The feedforward term n ff can be thought of as a set of nominal torque* which linearize 
the dynamics (!) about the operating points qj, qj, and q^, Therefore, it Is reasonable 
to add the linear feedback terms K^q^-q) +X p {qj-q) the control for the linearized 
system, These feedforward term* can be computed off-line, since they are function of 
the parameters of the desired trajectory only. 

On the other hand, the computed torque controller computes the dynamics on-line, 
using the sampled joint position and velocity data. The control equation Is: 

a„(^, q, fe, q, <U) - i(q)cf + V(q, q) + <5(q] + F(q,q) (i) 
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where q’ la given by, 


q' =r q+K y (q d - q] + K*(qj - q)> (G) 

If the robot model is exact, then each link of the robot is decoupled, and the trajectory 
error goes to 2 ero. Gilbert and Ha (1984) have shown chat the computed torque control 
method is robust to small modelling errors. 

Previously, r.Legeoia, Fournier, and Aid on (19S0) suggested feedforward control as 
an alternative to the on-line computation requirements of computed torque control, al¬ 
though they did not present any experimental results. Golla, Garg, and Hughes (L981) 
discussed different linear state-feedback controller using a linearized model of a ma¬ 
nipulator. Asada, Kanads, and Takeyama (1983) presented some results of applying a 
feedforward control to the early version of a direct drive arm at the Robotics Institute 
of CMIT, though for quite slow movements and for inertial parameters derived by CAD 
modeling. The computed torque method has been considered by several other inves¬ 
tigator* (Paul, 1972; Markiewics, 1973; Bejczy, 1974; Luh, Walker, and Paul, 1980}.. 
Although simulation results have been presented, there has been no published report on 
the actual implementation of this controller, mainly due to the Lack of an appropriate 
manipulator or on-line computational facility. 

In this paper, we first use the feedforward controller to evaluate the accuracy of 
our estimates of the link inertial parameters, and to compare its performance against 
several other simpler control methods for high speed movements. Then, we present 
some preliminary results on the implementation of a computed torque controller, again 
using the estimated inertial parameters of the links. 

1,2 Estimates of inertial parameters 

The Inertia] parameters in the feedforward computation for the SLDDA were estimated 
by an algorithm developed previously (An, Atkeson, and Holler bach, 19S5-), It was 
shown that the unknown inertial parameters of each link (mass, center of mass, and 
momenta af inertia} appear linearly In the rigid body dynamics of a manipulator. Then 
a least squares algorithm ww used to compute the estimates of these parameters. 

The accuracy of the inertial parameters was verified initially by comparing the mea- 
■uLured joint torques to the torques computed from the estimated parameters. This 
comparison* together with the torques computed from parameters derived by CAD 
modelling, Is shown in Figure I for a l-3s trajectory of all the joints moving 2§Q de¬ 
grees. The results were actually superior for the dynamically estimated parameters 
than for the CAD'modelled parameters, A more practical verification of the estimated 
parameters is in generating feedforward torques as part of a control algorithm. The 
results of such experiments- are presented in the next section, 

2 Robot Controller Experiments 

In this section, performances ofseveraL different controllers for full motion of the SLDDA 
are evaluated using the hybrid controller- The reference positions and velocities, and the 
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Figure 1: The measuredCAD-modelled, and eetimated joint torque. 

feedforward torque* are generated by a microprocessor and input tq three independent 
joint analog servos, We present evaluations of the following five control methods used 
for high speed movements of all three joint* of the manipulator: 

i. PD controller with position reference only: 

m = -M + K^i-q) 

2r PD controller with position reference and feedforward of gravity torques: 

a = G(qj] - K„q + K,(qj - q) 

3. PD controller with position and velocity references: 

a = K p {qs - q) + K,(qj - q) 

FD controller with position and velocity references plus feedforward of gravity 
torques: 


el - 6(q*J + K^fqj - 4) + K p(qtf - q) 













Figure 2: Trajectory errors of the 5 controllers for full 1.3* motion. 

5, PD controller with pomtioji and velocity references plus feedforward of full dy- 
n amice: 


* - + V(<u,q,) + 6(qj) + K,[qj - ^),+ K,,(q*- q) 

In. these experiments, friction wan neglected. The nominal position and velocity fains 
were adjusted experimentally to achieve high stiffheM and overdamped charateriatica 
without the feedforward terms. 

A fifth order polynomial tn joint apace ww used to generate the reference trajec¬ 
tory- The joints moved from (90% 269,1%-30 ? ] to (330*, 19.1% 220*) in 1.3a f with peat 
velocities of 360 dep/aec end the peak accelerations of 954 dtgfatt? far each joint- For 
control methods {2J„ (4), and (5) s the estimates of the link inertia] parameters given by 
An, AtkeaoiL, and Hollerbach {1995] ware used to compute the feedforward torques. 

The trajectory error* for the above 5 controllers are shown on Figure % The errors 
for the first controller are very large and are out of the graph range. Adding a gravity 
feedforward term does not help very much* and the trajectory error* for Controller 2 
are also very large. This was expected since gravity feedforward is a static correction to 
Controller l t and the dynamic effects dominate the response for high speed movements. 
Modifying the first controller by adding a velocity reference aignai improved the response 
greatly. As with Controller 2, adding a gravity feedforward term did not reduce the 
trajectory errors very much, and influenced mainly the steady state error* for joints 2 
and 3. 


















Figure 3; Trajectory errors of the three digital controllc» for lull 1.34 motion, 

The full feedforward controller reduced the trajectory errors significantly for joints 1 
and 2, with peat errors of only 0.33* and 0.64% respectively. For joint 3, the feedforward 
torques did not kelp because of the light inertia and the dominance of immodelled 
dynamics ia the motor and in bearing friction. The high feedback gains make this joint 
somewhat robust to these tmmodeiled dynamics;; yet t the trajectory errors could not be 
reduced below 1,4* with the feedforward torques based on the ideal rigid body model 
of the link. 

2.1 Computed Torque Controller Experiment 

In this section, some preliminary reeuita are presented for the computed torque method 
Implemented on the SLBDA. In this implementation, the analog servos are disabled, 
and the feedback computation is done digitally by one Motorola 68000 based micropro¬ 
cessor using scaled fixed-point arithmetic. Written in the C language, the contrailsr, 
including the full computation of the robot dyamics, runs at a 133 Hi sampling fre¬ 
quency. Although further improvements in computation time are possible, this speed 
was adequate in demonstrating the efficacy of dynamic compensation- The details of 
this implementation are discussed in [Griffiths,, 1966), 

A similar fifth order polynomial trajectory as in the previous section was used for 
this experiment. Figure 3 shows the trajectory errors for three controllers] the digital 
FD controller, the feedforward controller using a digital servo, and the computed torque 
controller. The computed torque and the feedforward controllers both show a significant 
















reduction in tracking errois for joints l and -2 compared with the PD control;, with no 
clear distinction between feedforward and computed torque. The tracking errors for 
joint 1 range from 4.4 fl to 2,2* and for joint 2 go from 3,5 fl to 2.0" with the addition of 
dynamic component. As before, the trajectory errora for joint 3 were not reduced by 
the computed torque or the feedforward controller. Again, this aeema to indicate that 
our model for the third link, (nay not be very good. 

The trajectory' tracking performance oF the computed torque controller is not as 
good as that of the analog feedforward controller of the previous section. The main 
reason for this in the slow sampling frequency (133 Ha) of the digital controller, as 
compared to the 1 KHz sampling frequency at which the reference inputs were given 
to the analog servos. Improvements In the computation, time should also improve the 
tracking performance of the computed torque controller, 

3 Conclusions 

We have presented experimental results of using an estimated dynamic model of the 
manipulator for dynamic compensation via feedforward and computed torque control 
methods. The results indicate that dynamic compensation can improve trajectory accu¬ 
racy significantly and that the estimated rigid body model of the manipulator is quite 
accurate and adequate for control purposes for joints 1 and 2. The umnodelled dynam¬ 
ics of the light third link, including the motor dynamics and friction, are dominant and 
yield larger trajectory errors than at the other two jointe. Therefore, for joint 3, it may 
be necessary to use a more complete model to improve trajectory following. 

The results of the digital implementation, of the feedforward and computed torque 
controllers were not as good as the hybrid feedforward controller. This indicates that 
if a robot was being used solely for free space movements without significant variation 
of its loads, then a hybrid controller using an independent analog servo for each joint 
may be quite adequate, A hybrid controller, however, is not flexible, and cannot handle 
varying loads oe interactions with the environment. Future experiments with the MiT 
Serial Link Direct Drive Arm will concentrate on improving the computation, time for 
the digital control system and on issues of force control, 
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